2026-02-17T11:55:11.388 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T11:55:11.403 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T11:55:11.480 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51384 branch: wip-bharath20-testing-2026-02-13-0906-squid description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '51384' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath20-testing-2026-02-13-0906-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-bharath20-testing-2026-02-13-0906-squid sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 4581 sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath20-testing-2026-02-13-0906-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 targets: trial044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuYWHiqFWt+8oh9m/WiS1U7IyjMUO9ms+dYUqoVnHNlcU7KRfHDau5aGVfhihB16BdACMhiuBzyY1n5RdMlE2I= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_04:10:36 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T11:55:11.480 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa; will attempt to use it 2026-02-17T11:55:11.481 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa/tasks 2026-02-17T11:55:11.481 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T11:55:11.483 INFO:teuthology.task.internal:Checking packages... 2026-02-17T11:55:11.483 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3c96dfb3f50b3d02598a82af7104764bfdcd3919' 2026-02-17T11:55:11.483 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T11:55:11.483 INFO:teuthology.packaging:ref: None 2026-02-17T11:55:11.483 INFO:teuthology.packaging:tag: None 2026-02-17T11:55:11.483 INFO:teuthology.packaging:branch: wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T11:55:11.483 INFO:teuthology.packaging:sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:55:11.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T11:55:11.614 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-675.g3c96dfb3 2026-02-17T11:55:11.615 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T11:55:11.627 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T11:55:11.627 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T11:55:11.642 INFO:teuthology.task.internal:Saving configuration 2026-02-17T11:55:11.651 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T11:55:11.667 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T11:55:11.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial044.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51384', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 11:51:45.866992', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuYWHiqFWt+8oh9m/WiS1U7IyjMUO9ms+dYUqoVnHNlcU7KRfHDau5aGVfhihB16BdACMhiuBzyY1n5RdMlE2I='} 2026-02-17T11:55:11.747 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T11:55:11.759 INFO:teuthology.task.internal:roles: ubuntu@trial044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T11:55:11.759 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T11:55:11.843 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x70445f4bd760>, signals=[15]) 2026-02-17T11:55:11.843 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T11:55:11.854 INFO:teuthology.task.internal:Opening connections... 2026-02-17T11:55:11.854 DEBUG:teuthology.task.internal:connecting to ubuntu@trial044.front.sepia.ceph.com 2026-02-17T11:55:11.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:55:11.919 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T11:55:11.930 DEBUG:teuthology.orchestra.run.trial044:> uname -m 2026-02-17T11:55:11.992 INFO:teuthology.orchestra.run.trial044.stdout:x86_64 2026-02-17T11:55:11.992 DEBUG:teuthology.orchestra.run.trial044:> cat /etc/os-release 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:NAME="CentOS Stream" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:VERSION="9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:ID="centos" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:ID_LIKE="rhel fedora" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:VERSION_ID="9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:PLATFORM_ID="platform:el9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:ANSI_COLOR="0;31" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:LOGO="fedora-logo-icon" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:HOME_URL="https://centos.org/" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T11:55:12.047 INFO:teuthology.orchestra.run.trial044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T11:55:12.047 INFO:teuthology.lock.ops:Updating trial044.front.sepia.ceph.com on lock server 2026-02-17T11:55:12.110 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T11:55:12.124 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T11:55:12.135 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T11:55:12.136 DEBUG:teuthology.orchestra.run.trial044:> test '!' -e /home/ubuntu/cephtest 2026-02-17T11:55:12.151 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T11:55:12.162 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T11:55:12.162 DEBUG:teuthology.orchestra.run.trial044:> test -z $(ls -A /var/lib/ceph) 2026-02-17T11:55:12.207 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T11:55:12.293 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T11:55:12.314 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T11:55:12.314 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T11:55:12.314 DEBUG:teuthology.orchestra.run.trial044:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T11:55:12.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T11:55:12.329 DEBUG:teuthology.orchestra.run.trial044:> uname -r 2026-02-17T11:55:12.385 INFO:teuthology.orchestra.run.trial044.stdout:5.14.0-645.el9.x86_64 2026-02-17T11:55:12.385 INFO:teuthology.task.kernel:Running kernel on trial044: 5.14.0-645.el9.x86_64 2026-02-17T11:55:12.386 DEBUG:teuthology.orchestra.run.trial044:> sudo yum install -y kernel 2026-02-17T11:55:13.585 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-17T11:55:16.582 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T11:55:29.081 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - CRB 796 kB/s | 7.8 MB 00:09 2026-02-17T11:55:30.519 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 20 kB 00:00 2026-02-17T11:55:31.894 INFO:teuthology.orchestra.run.trial044.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2026-02-17T11:55:35.094 INFO:teuthology.orchestra.run.trial044.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-17T11:55:36.102 INFO:teuthology.orchestra.run.trial044.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T11:55:36.122 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T11:55:36.123 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:Install 4 Packages 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 88 M 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 135 M 2026-02-17T11:55:36.124 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T11:55:36.319 INFO:teuthology.orchestra.run.trial044.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 6.0 MB/s | 803 kB 00:00 2026-02-17T11:55:36.468 INFO:teuthology.orchestra.run.trial044.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 61 MB/s | 17 MB 00:00 2026-02-17T11:55:36.867 INFO:teuthology.orchestra.run.trial044.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 56 MB/s | 31 MB 00:00 2026-02-17T11:55:36.989 INFO:teuthology.orchestra.run.trial044.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2026-02-17T11:55:36.990 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T11:55:36.990 INFO:teuthology.orchestra.run.trial044.stdout:Total 102 MB/s | 88 MB 00:00 2026-02-17T11:55:37.181 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T11:55:37.378 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T11:55:37.378 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T11:55:37.724 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T11:55:37.725 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T11:55:38.803 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T11:55:39.085 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T11:55:39.105 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T11:55:40.889 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T11:55:40.916 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T11:55:43.935 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T11:55:43.941 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:55:46.974 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:56:17.286 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:56:17.308 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:56:17.373 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:56:17.373 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T11:56:17.373 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T11:56:17.373 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.426 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.427 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:56:17.427 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T11:56:17.482 DEBUG:teuthology.orchestra.run.trial044:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T11:56:17.508 INFO:teuthology.orchestra.run.trial044.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.509 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T11:56:17.509 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T11:56:17.509 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T11:56:17.509 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T11:56:17.509 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T11:56:17.509 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial044.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T11:56:17.509 DEBUG:teuthology.orchestra.run.trial044:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T11:56:17.572 INFO:teuthology.orchestra.run.trial044.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.574 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T11:56:17.574 DEBUG:teuthology.orchestra.run.trial044:> sudo rpm -qi grub2-tools 2026-02-17T11:56:17.641 INFO:teuthology.orchestra.run.trial044.stdout:Name : grub2-tools 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Epoch : 1 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Version : 2.06 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Release : 118.el9 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Architecture: x86_64 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Group : Unspecified 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Size : 8393909 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:License : GPLv3+ 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T11:56:17.642 INFO:teuthology.orchestra.run.trial044.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Packager : builder@centos.org 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Vendor : CentOS 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Summary : Support tools for GRUB. 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:Description : 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T11:56:17.643 INFO:teuthology.orchestra.run.trial044.stdout:hardware devices. 2026-02-17T11:56:17.644 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T11:56:17.644 INFO:teuthology.orchestra.run.trial044.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T11:56:17.645 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T11:56:17.645 INFO:teuthology.task.kernel:Updating grub on trial044 to boot 5.14.0-681.el9.x86_64 2026-02-17T11:56:17.645 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T11:56:17.711 INFO:teuthology.orchestra.run.trial044.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T11:56:17.712 DEBUG:teuthology.orchestra.run.trial044:> sudo command -v grubby && echo yes || echo no 2026-02-17T11:56:17.775 INFO:teuthology.orchestra.run.trial044.stdout:/sbin/grubby 2026-02-17T11:56:17.777 INFO:teuthology.orchestra.run.trial044.stdout:yes 2026-02-17T11:56:17.777 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T11:56:17.845 INFO:teuthology.orchestra.run.trial044.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T11:56:17.846 DEBUG:teuthology.orchestra.run.trial044:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T11:56:17.918 DEBUG:teuthology.orchestra.run.trial044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T11:56:18.124 INFO:teuthology.orchestra.run.trial044.stderr:Generating grub configuration file ... 2026-02-17T11:56:18.187 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T11:56:18.194 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T11:56:18.441 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T11:56:18.447 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T11:56:18.479 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T11:56:18.485 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T11:56:18.508 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T11:56:18.515 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T11:56:18.596 INFO:teuthology.orchestra.run.trial044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T11:56:18.603 INFO:teuthology.orchestra.run.trial044.stderr:done 2026-02-17T11:56:18.605 DEBUG:teuthology.orchestra.run.trial044:> test -d /sys/firmware/efi 2026-02-17T11:56:18.620 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T11:56:18.685 INFO:teuthology.orchestra.run.trial044.stdout:/boot/efi/EFI/BOOT 2026-02-17T11:56:18.685 INFO:teuthology.orchestra.run.trial044.stdout:/boot/efi/EFI/centos 2026-02-17T11:56:18.687 DEBUG:teuthology.orchestra.run.trial044:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T11:56:18.752 INFO:teuthology.orchestra.run.trial044.stdout:no 2026-02-17T11:56:18.752 DEBUG:teuthology.orchestra.run.trial044:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T11:56:18.816 INFO:teuthology.orchestra.run.trial044.stdout:yes 2026-02-17T11:56:18.817 DEBUG:teuthology.orchestra.run.trial044:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T11:56:18.882 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial044 2026-02-17T11:56:18.882 DEBUG:teuthology.orchestra.run.trial044:> sudo bash -c ' 2026-02-17T11:56:18.883 DEBUG:teuthology.orchestra.run.trial044:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T11:56:18.883 DEBUG:teuthology.orchestra.run.trial044:> ' 2026-02-17T11:56:48.925 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T11:56:48.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-17T11:56:48.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:57:03.232 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.44 2026-02-17T11:57:12.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-17T11:57:12.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:57:48.419 DEBUG:teuthology.orchestra.run.trial044:> true 2026-02-17T11:57:48.580 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-17T11:57:48.580 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T11:57:48.580 DEBUG:teuthology.orchestra.run.trial044:> uname -r 2026-02-17T11:57:48.636 INFO:teuthology.orchestra.run.trial044.stdout:5.14.0-681.el9.x86_64 2026-02-17T11:57:48.636 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T11:57:48.636 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T11:57:48.636 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T11:57:49.637 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T11:57:49.637 DEBUG:teuthology.orchestra.run.trial044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T11:57:49.666 INFO:teuthology.orchestra.run.trial044.stdout:ttyS1 2026-02-17T11:57:49.686 DEBUG:teuthology.parallel:result is None 2026-02-17T11:57:49.686 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T11:57:49.697 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T11:57:49.697 DEBUG:teuthology.orchestra.run.trial044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T11:57:49.743 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T11:57:49.755 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T11:57:49.769 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T11:57:49.769 DEBUG:teuthology.orchestra.run.trial044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T11:57:49.806 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T11:57:49.820 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T11:57:49.821 DEBUG:teuthology.orchestra.run.trial044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T11:57:49.866 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:57:49.875 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:57:49.877 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T11:57:49.888 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T11:57:49.889 DEBUG:teuthology.orchestra.run.trial044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T11:57:49.942 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T11:57:49.959 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.trial044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T11:57:49.999 DEBUG:teuthology.orchestra.run.trial044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T11:57:50.160 DEBUG:teuthology.orchestra.run.trial044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T11:57:50.185 DEBUG:teuthology.orchestra.run.trial044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T11:57:50.333 DEBUG:teuthology.orchestra.run.trial044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T11:57:50.357 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T11:57:50.357 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T11:57:50.420 DEBUG:teuthology.orchestra.run.trial044:> sudo service rsyslog restart 2026-02-17T11:57:50.489 INFO:teuthology.orchestra.run.trial044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T11:57:50.883 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T11:57:50.896 INFO:teuthology.task.internal:Starting timer... 2026-02-17T11:57:50.896 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T11:57:50.917 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T11:57:50.936 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-17T11:57:50.936 DEBUG:teuthology.orchestra.run.trial044:> sudo service auditd rotate 2026-02-17T11:57:50.964 INFO:teuthology.orchestra.run.trial044.stdout:Rotating logs: 2026-02-17T11:57:50.967 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T11:57:50.967 DEBUG:teuthology.orchestra.run.trial044:> /usr/sbin/getenforce 2026-02-17T11:57:51.025 INFO:teuthology.orchestra.run.trial044.stdout:Permissive 2026-02-17T11:57:51.025 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial044.front.sepia.ceph.com': 'permissive'} 2026-02-17T11:57:51.026 DEBUG:teuthology.orchestra.run.trial044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T11:57:51.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T11:57:51.091 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T11:57:51.091 DEBUG:teuthology.orchestra.run.trial044:> sudo /usr/sbin/setenforce permissive 2026-02-17T11:57:51.154 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T11:57:51.174 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T11:57:51.184 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T11:57:51.186 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial044.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T12:00:41.484 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial044.front.sepia.ceph.com')] 2026-02-17T12:00:41.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-17T12:00:41.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T12:00:41.589 DEBUG:teuthology.orchestra.run.trial044:> true 2026-02-17T12:00:41.665 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-17T12:00:41.665 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T12:00:41.678 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T12:00:41.678 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T12:00:41.679 DEBUG:teuthology.orchestra.run.trial044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T12:00:41.734 INFO:teuthology.orchestra.run.trial044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T12:00:41.746 INFO:teuthology.orchestra.run.trial044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T12:00:41.767 INFO:teuthology.orchestra.run.trial044.stderr:sudo: ntpd: command not found 2026-02-17T12:00:41.777 INFO:teuthology.orchestra.run.trial044.stdout:506 Cannot talk to daemon 2026-02-17T12:00:41.788 INFO:teuthology.orchestra.run.trial044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T12:00:41.800 INFO:teuthology.orchestra.run.trial044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T12:00:41.844 INFO:teuthology.orchestra.run.trial044.stderr:bash: line 1: ntpq: command not found 2026-02-17T12:00:41.915 INFO:teuthology.orchestra.run.trial044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T12:00:41.915 INFO:teuthology.orchestra.run.trial044.stdout:=============================================================================== 2026-02-17T12:00:41.915 INFO:teuthology.orchestra.run.trial044.stdout:^? 216.82.35.115 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T12:00:41.915 INFO:teuthology.orchestra.run.trial044.stdout:^? ntp2.wiktel.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T12:00:41.915 INFO:teuthology.orchestra.run.trial044.stdout:^? kc1cloud.147.72.155.23.m> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T12:00:41.916 INFO:teuthology.orchestra.run.trial044.stdout:^? 1636541-static.urdliaaa.> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T12:00:41.917 INFO:teuthology.run_tasks:Running task install... 2026-02-17T12:00:41.933 DEBUG:teuthology.task.install:project ceph 2026-02-17T12:00:41.933 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3c96dfb3f50b3d02598a82af7104764bfdcd3919'}} 2026-02-17T12:00:41.933 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3c96dfb3f50b3d02598a82af7104764bfdcd3919'} 2026-02-17T12:00:41.933 INFO:teuthology.task.install:Using flavor: default 2026-02-17T12:00:41.936 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T12:00:41.936 INFO:teuthology.task.install:extra packages: [] 2026-02-17T12:00:41.936 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3c96dfb3f50b3d02598a82af7104764bfdcd3919', 'tag': None, 'wait_for_package': False} 2026-02-17T12:00:41.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T12:00:42.061 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/centos/9/flavors/default/ 2026-02-17T12:00:42.061 INFO:teuthology.task.install.rpm:Package version is 19.2.3-675.g3c96dfb3 2026-02-17T12:00:42.193 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T12:00:42.193 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:00:42.193 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T12:00:42.218 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T12:00:42.218 DEBUG:teuthology.orchestra.run.trial044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3c96dfb3f50b3d02598a82af7104764bfdcd3919/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T12:00:42.284 DEBUG:teuthology.orchestra.run.trial044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T12:00:42.358 DEBUG:teuthology.orchestra.run.trial044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T12:00:42.422 INFO:teuthology.orchestra.run.trial044.stdout:check_obsoletes = 1 2026-02-17T12:00:42.424 DEBUG:teuthology.orchestra.run.trial044:> sudo yum clean all 2026-02-17T12:00:42.575 INFO:teuthology.orchestra.run.trial044.stdout:41 files removed 2026-02-17T12:00:42.590 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-radosgw 2026-02-17T12:00:42.857 INFO:teuthology.orchestra.run.trial044.stdout:ceph packages for x86_64 689 kB/s | 86 kB 00:00 2026-02-17T12:00:42.987 INFO:teuthology.orchestra.run.trial044.stdout:ceph noarch packages 114 kB/s | 12 kB 00:00 2026-02-17T12:00:43.111 INFO:teuthology.orchestra.run.trial044.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-17T12:00:44.483 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.9 MB 00:01 2026-02-17T12:00:47.785 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 27 MB 00:02 2026-02-17T12:00:51.336 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 7.8 MB 00:01 2026-02-17T12:00:53.194 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-17T12:00:53.838 INFO:teuthology.orchestra.run.trial044.stdout:Extra Packages for Enterprise Linux 35 MB/s | 20 MB 00:00 2026-02-17T12:00:57.209 INFO:teuthology.orchestra.run.trial044.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-17T12:00:58.426 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:00:58.427 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:00:58.427 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: ceph-radosgw x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 11 M 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout:Upgrading: 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: librados2 x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 3.4 M 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: librbd1 x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 3.2 M 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: ceph-base x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 5.5 M 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: ceph-common x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 22 M 2026-02-17T12:00:58.428 INFO:teuthology.orchestra.run.trial044.stdout: ceph-selinux x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 25 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs2 x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 999 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libradosstriper1 x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 503 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: librgw2 x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 5.4 M 2026-02-17T12:00:58.429 INFO:teuthology.orchestra.run.trial044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-argparse x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 45 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-common x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 142 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-cephfs x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 165 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-rados x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 322 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-rbd x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 303 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: python3-rgw x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 100 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T12:00:58.430 INFO:teuthology.orchestra.run.trial044.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout:Installing weak dependencies: 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout:Install 33 Packages 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout:Upgrade 2 Packages 2026-02-17T12:00:58.431 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:00:58.432 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 63 M 2026-02-17T12:00:58.432 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:00:59.200 INFO:teuthology.orchestra.run.trial044.stdout:(1/35): ceph-base-19.2.3-675.g3c96dfb3.el9.x86_ 18 MB/s | 5.5 MB 00:00 2026-02-17T12:00:59.235 INFO:teuthology.orchestra.run.trial044.stdout:(2/35): ceph-selinux-19.2.3-675.g3c96dfb3.el9.x 705 kB/s | 25 kB 00:00 2026-02-17T12:00:59.295 INFO:teuthology.orchestra.run.trial044.stdout:(3/35): libcephfs2-19.2.3-675.g3c96dfb3.el9.x86 17 MB/s | 999 kB 00:00 2026-02-17T12:00:59.346 INFO:teuthology.orchestra.run.trial044.stdout:(4/35): libradosstriper1-19.2.3-675.g3c96dfb3.e 9.5 MB/s | 503 kB 00:00 2026-02-17T12:00:59.562 INFO:teuthology.orchestra.run.trial044.stdout:(5/35): ceph-radosgw-19.2.3-675.g3c96dfb3.el9.x 16 MB/s | 11 MB 00:00 2026-02-17T12:00:59.589 INFO:teuthology.orchestra.run.trial044.stdout:(6/35): librgw2-19.2.3-675.g3c96dfb3.el9.x86_64 22 MB/s | 5.4 MB 00:00 2026-02-17T12:00:59.589 INFO:teuthology.orchestra.run.trial044.stdout:(7/35): python3-ceph-argparse-19.2.3-675.g3c96d 1.6 MB/s | 45 kB 00:00 2026-02-17T12:00:59.607 INFO:teuthology.orchestra.run.trial044.stdout:(8/35): python3-ceph-common-19.2.3-675.g3c96dfb 7.7 MB/s | 142 kB 00:00 2026-02-17T12:00:59.612 INFO:teuthology.orchestra.run.trial044.stdout:(9/35): python3-cephfs-19.2.3-675.g3c96dfb3.el9 7.1 MB/s | 165 kB 00:00 2026-02-17T12:00:59.631 INFO:teuthology.orchestra.run.trial044.stdout:(10/35): python3-rados-19.2.3-675.g3c96dfb3.el9 13 MB/s | 322 kB 00:00 2026-02-17T12:00:59.638 INFO:teuthology.orchestra.run.trial044.stdout:(11/35): python3-rbd-19.2.3-675.g3c96dfb3.el9.x 11 MB/s | 303 kB 00:00 2026-02-17T12:00:59.653 INFO:teuthology.orchestra.run.trial044.stdout:(12/35): python3-rgw-19.2.3-675.g3c96dfb3.el9.x 4.4 MB/s | 100 kB 00:00 2026-02-17T12:00:59.721 INFO:teuthology.orchestra.run.trial044.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 403 kB/s | 33 kB 00:00 2026-02-17T12:00:59.795 INFO:teuthology.orchestra.run.trial044.stdout:(14/35): ceph-common-19.2.3-675.g3c96dfb3.el9.x 24 MB/s | 22 MB 00:00 2026-02-17T12:00:59.795 INFO:teuthology.orchestra.run.trial044.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 1.2 MB/s | 93 kB 00:00 2026-02-17T12:00:59.884 INFO:teuthology.orchestra.run.trial044.stdout:(16/35): nvme-cli-2.13-1.el9.x86_64.rpm 4.7 MB/s | 1.1 MB 00:00 2026-02-17T12:01:00.738 INFO:teuthology.orchestra.run.trial044.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 174 kB/s | 164 kB 00:00 2026-02-17T12:01:00.745 INFO:teuthology.orchestra.run.trial044.stdout:(18/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 186 kB/s | 160 kB 00:00 2026-02-17T12:01:00.830 INFO:teuthology.orchestra.run.trial044.stdout:(19/35): boost-program-options-1.75.0-13.el9.x8 100 kB/s | 104 kB 00:01 2026-02-17T12:01:00.839 INFO:teuthology.orchestra.run.trial044.stdout:(20/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 445 kB/s | 45 kB 00:00 2026-02-17T12:01:01.183 INFO:teuthology.orchestra.run.trial044.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 696 kB/s | 240 kB 00:00 2026-02-17T12:01:01.285 INFO:teuthology.orchestra.run.trial044.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 364 kB/s | 37 kB 00:00 2026-02-17T12:01:01.386 INFO:teuthology.orchestra.run.trial044.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 655 kB/s | 66 kB 00:00 2026-02-17T12:01:01.584 INFO:teuthology.orchestra.run.trial044.stdout:(24/35): librdkafka-1.6.1-102.el9.x86_64.rpm 790 kB/s | 662 kB 00:00 2026-02-17T12:01:01.620 INFO:teuthology.orchestra.run.trial044.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-17T12:01:01.830 INFO:teuthology.orchestra.run.trial044.stdout:(26/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 292 kB/s | 292 kB 00:01 2026-02-17T12:01:01.855 INFO:teuthology.orchestra.run.trial044.stdout:(27/35): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-17T12:01:01.855 INFO:teuthology.orchestra.run.trial044.stdout:(28/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 105 kB/s | 25 kB 00:00 2026-02-17T12:01:01.863 INFO:teuthology.orchestra.run.trial044.stdout:(29/35): liboath-2.6.12-1.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2026-02-17T12:01:01.888 INFO:teuthology.orchestra.run.trial044.stdout:(30/35): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2026-02-17T12:01:01.927 INFO:teuthology.orchestra.run.trial044.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2026-02-17T12:01:01.935 INFO:teuthology.orchestra.run.trial044.stdout:(32/35): re2-20211101-20.el9.x86_64.rpm 2.6 MB/s | 191 kB 00:00 2026-02-17T12:01:02.008 INFO:teuthology.orchestra.run.trial044.stdout:(33/35): thrift-0.15.0-4.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2026-02-17T12:01:02.080 INFO:teuthology.orchestra.run.trial044.stdout:(34/35): librados2-19.2.3-675.g3c96dfb3.el9.x86 22 MB/s | 3.4 MB 00:00 2026-02-17T12:01:02.087 INFO:teuthology.orchestra.run.trial044.stdout:(35/35): librbd1-19.2.3-675.g3c96dfb3.el9.x86_6 21 MB/s | 3.2 MB 00:00 2026-02-17T12:01:02.091 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:02.091 INFO:teuthology.orchestra.run.trial044.stdout:Total 17 MB/s | 63 MB 00:03 2026-02-17T12:01:02.188 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:02.206 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:02.207 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:02.443 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:02.444 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:02.735 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:02.743 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T12:01:02.746 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T12:01:02.757 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T12:01:02.908 INFO:teuthology.orchestra.run.trial044.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T12:01:02.912 INFO:teuthology.orchestra.run.trial044.stdout: Upgrading : librados2-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/37 2026-02-17T12:01:02.943 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados2-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/37 2026-02-17T12:01:02.986 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rados-2:19.2.3-675.g3c96dfb3.el9.x86_64 6/37 2026-02-17T12:01:02.987 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs2-2:19.2.3-675.g3c96dfb3.el9.x86_64 7/37 2026-02-17T12:01:02.999 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephfs2-2:19.2.3-675.g3c96dfb3.el9.x86_64 7/37 2026-02-17T12:01:03.003 INFO:teuthology.orchestra.run.trial044.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T12:01:03.012 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ceph-argparse-2:19.2.3-675.g3c96dfb3.el9.x 9/37 2026-02-17T12:01:03.034 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cephfs-2:19.2.3-675.g3c96dfb3.el9.x86_64 10/37 2026-02-17T12:01:03.035 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libradosstriper1-2:19.2.3-675.g3c96dfb3.el9.x86_64 11/37 2026-02-17T12:01:03.047 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libradosstriper1-2:19.2.3-675.g3c96dfb3.el9.x86_64 11/37 2026-02-17T12:01:03.078 INFO:teuthology.orchestra.run.trial044.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T12:01:03.081 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T12:01:03.088 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T12:01:03.091 INFO:teuthology.orchestra.run.trial044.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T12:01:03.094 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T12:01:03.099 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T12:01:03.238 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T12:01:03.239 INFO:teuthology.orchestra.run.trial044.stdout: Upgrading : librbd1-2:19.2.3-675.g3c96dfb3.el9.x86_64 19/37 2026-02-17T12:01:03.265 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librbd1-2:19.2.3-675.g3c96dfb3.el9.x86_64 19/37 2026-02-17T12:01:03.268 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 20/37 2026-02-17T12:01:03.274 INFO:teuthology.orchestra.run.trial044.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T12:01:03.499 INFO:teuthology.orchestra.run.trial044.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T12:01:03.500 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librgw2-2:19.2.3-675.g3c96dfb3.el9.x86_64 23/37 2026-02-17T12:01:03.516 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librgw2-2:19.2.3-675.g3c96dfb3.el9.x86_64 23/37 2026-02-17T12:01:03.519 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 24/37 2026-02-17T12:01:03.523 INFO:teuthology.orchestra.run.trial044.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T12:01:03.525 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T12:01:03.526 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T12:01:03.545 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T12:01:03.663 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T12:01:03.665 INFO:teuthology.orchestra.run.trial044.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T12:01:03.943 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T12:01:03.943 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T12:01:03.943 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:03.979 INFO:teuthology.orchestra.run.trial044.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T12:01:03.981 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86 31/37 2026-02-17T12:01:04.959 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 32/37 2026-02-17T12:01:04.962 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 32/37 2026-02-17T12:01:05.242 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 32/37 2026-02-17T12:01:05.245 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-base-2:19.2.3-675.g3c96dfb3.el9.x86_64 33/37 2026-02-17T12:01:05.287 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-base-2:19.2.3-675.g3c96dfb3.el9.x86_64 33/37 2026-02-17T12:01:05.287 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T12:01:05.287 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T12:01:05.287 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:05.291 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-selinux-2:19.2.3-675.g3c96dfb3.el9.x86_64 34/37 2026-02-17T12:01:09.825 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-selinux-2:19.2.3-675.g3c96dfb3.el9.x86_64 34/37 2026-02-17T12:01:09.825 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /sys 2026-02-17T12:01:09.825 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /proc 2026-02-17T12:01:09.825 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /mnt 2026-02-17T12:01:09.825 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /var/tmp 2026-02-17T12:01:09.826 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /home 2026-02-17T12:01:09.826 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /root 2026-02-17T12:01:09.826 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /tmp 2026-02-17T12:01:09.826 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:10.281 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-radosgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 35/37 2026-02-17T12:01:10.304 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 35/37 2026-02-17T12:01:10.304 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:10.305 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T12:01:10.305 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T12:01:10.305 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T12:01:10.305 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:10.307 INFO:teuthology.orchestra.run.trial044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T12:01:10.320 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T12:01:10.320 INFO:teuthology.orchestra.run.trial044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T12:01:10.376 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T12:01:10.376 INFO:teuthology.orchestra.run.trial044.stdout:Creating group 'qat' with GID 992. 2026-02-17T12:01:10.376 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-base-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 2/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-radosgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 3/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-selinux-2:19.2.3-675.g3c96dfb3.el9.x86_64 4/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs2-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libradosstriper1-2:19.2.3-675.g3c96dfb3.el9.x86_64 6/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librgw2-2:19.2.3-675.g3c96dfb3.el9.x86_64 7/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ceph-argparse-2:19.2.3-675.g3c96dfb3.el9.x 8/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86 9/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cephfs-2:19.2.3-675.g3c96dfb3.el9.x86_64 10/37 2026-02-17T12:01:10.646 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rados-2:19.2.3-675.g3c96dfb3.el9.x86_64 11/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 12/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 13/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T12:01:10.647 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T12:01:10.648 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados2-2:19.2.3-675.g3c96dfb3.el9.x86_64 34/37 2026-02-17T12:01:10.648 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T12:01:10.648 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librbd1-2:19.2.3-675.g3c96dfb3.el9.x86_64 36/37 2026-02-17T12:01:10.707 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout:Upgraded: 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: librados2-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: librbd1-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: ceph-base-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: ceph-radosgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.708 INFO:teuthology.orchestra.run.trial044.stdout: ceph-selinux-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs2-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T12:01:10.709 INFO:teuthology.orchestra.run.trial044.stdout: libradosstriper1-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: librgw2-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-argparse-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-common-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.710 INFO:teuthology.orchestra.run.trial044.stdout: python3-cephfs-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: python3-rados-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: python3-rbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: python3-rgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:10.711 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:10.752 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-test 2026-02-17T12:01:11.058 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:14 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:11.140 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout: ceph-test x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 50 M 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout: libcephsqlite x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 163 k 2026-02-17T12:01:11.142 INFO:teuthology.orchestra.run.trial044.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:Install 5 Packages 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 50 M 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 212 M 2026-02-17T12:01:11.143 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:11.301 INFO:teuthology.orchestra.run.trial044.stdout:(1/5): libcephsqlite-19.2.3-675.g3c96dfb3.el9.x 1.7 MB/s | 163 kB 00:00 2026-02-17T12:01:11.999 INFO:teuthology.orchestra.run.trial044.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 294 kB/s | 233 kB 00:00 2026-02-17T12:01:12.137 INFO:teuthology.orchestra.run.trial044.stdout:(3/5): socat-1.7.4.1-8.el9.x86_64.rpm 363 kB/s | 303 kB 00:00 2026-02-17T12:01:12.235 INFO:teuthology.orchestra.run.trial044.stdout:(4/5): ceph-test-19.2.3-675.g3c96dfb3.el9.x86_6 48 MB/s | 50 MB 00:01 2026-02-17T12:01:12.236 INFO:teuthology.orchestra.run.trial044.stdout:(5/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 269 kB/s | 64 kB 00:00 2026-02-17T12:01:12.238 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:12.238 INFO:teuthology.orchestra.run.trial044.stdout:Total 46 MB/s | 50 MB 00:01 2026-02-17T12:01:12.266 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:12.275 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:12.276 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:12.460 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:12.461 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:12.675 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:12.681 INFO:teuthology.orchestra.run.trial044.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T12:01:12.686 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T12:01:12.694 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T12:01:12.695 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephsqlite-2:19.2.3-675.g3c96dfb3.el9.x86_64 4/5 2026-02-17T12:01:14.992 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephsqlite-2:19.2.3-675.g3c96dfb3.el9.x86_64 4/5 2026-02-17T12:01:15.010 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-test-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/5 2026-02-17T12:01:15.246 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-test-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/5 2026-02-17T12:01:15.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-test-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/5 2026-02-17T12:01:15.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephsqlite-2:19.2.3-675.g3c96dfb3.el9.x86_64 2/5 2026-02-17T12:01:15.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T12:01:15.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T12:01:15.299 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T12:01:15.299 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:15.299 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:15.299 INFO:teuthology.orchestra.run.trial044.stdout: ceph-test-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout: libcephsqlite-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:15.300 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:15.333 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph 2026-02-17T12:01:15.640 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:15.729 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout:====================================================================================== 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout:====================================================================================== 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout: ceph x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 6.5 k 2026-02-17T12:01:15.732 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-grafana-dashboards noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 31 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mds x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 2.4 M 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 1.1 M 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-modules-core noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 253 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mon x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 4.7 M 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 17 M 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: ceph-prometheus-alerts noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 17 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: cephadm noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 769 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T12:01:15.733 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T12:01:15.734 INFO:teuthology.orchestra.run.trial044.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T12:01:15.735 INFO:teuthology.orchestra.run.trial044.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T12:01:15.736 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T12:01:15.737 INFO:teuthology.orchestra.run.trial044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T12:01:15.738 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T12:01:15.739 INFO:teuthology.orchestra.run.trial044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout:Installing weak dependencies: 2026-02-17T12:01:15.740 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-cephadm noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 150 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-dashboard noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 3.8 M 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 7.4 M 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 22 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-rook noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 49 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: ceph-volume noarch 2:19.2.3-675.g3c96dfb3.el9 ceph-noarch 299 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout:====================================================================================== 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout:Install 99 Packages 2026-02-17T12:01:15.741 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:15.742 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 95 M 2026-02-17T12:01:15.742 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 520 M 2026-02-17T12:01:15.742 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:16.406 INFO:teuthology.orchestra.run.trial044.stdout:(1/99): ceph-19.2.3-675.g3c96dfb3.el9.x86_64.rp 121 kB/s | 6.5 kB 00:00 2026-02-17T12:01:16.471 INFO:teuthology.orchestra.run.trial044.stdout:(2/99): ceph-mgr-19.2.3-675.g3c96dfb3.el9.x86_6 9.0 MB/s | 1.1 MB 00:00 2026-02-17T12:01:16.540 INFO:teuthology.orchestra.run.trial044.stdout:(3/99): ceph-mds-19.2.3-675.g3c96dfb3.el9.x86_6 13 MB/s | 2.4 MB 00:00 2026-02-17T12:01:16.576 INFO:teuthology.orchestra.run.trial044.stdout:(4/99): ceph-grafana-dashboards-19.2.3-675.g3c9 845 kB/s | 31 kB 00:00 2026-02-17T12:01:16.618 INFO:teuthology.orchestra.run.trial044.stdout:(5/99): ceph-mgr-cephadm-19.2.3-675.g3c96dfb3.e 3.6 MB/s | 150 kB 00:00 2026-02-17T12:01:16.684 INFO:teuthology.orchestra.run.trial044.stdout:(6/99): ceph-mon-19.2.3-675.g3c96dfb3.el9.x86_6 17 MB/s | 4.7 MB 00:00 2026-02-17T12:01:16.839 INFO:teuthology.orchestra.run.trial044.stdout:(7/99): ceph-mgr-dashboard-19.2.3-675.g3c96dfb3 17 MB/s | 3.8 MB 00:00 2026-02-17T12:01:16.887 INFO:teuthology.orchestra.run.trial044.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-675.g3c96dfb3 458 kB/s | 22 kB 00:00 2026-02-17T12:01:16.942 INFO:teuthology.orchestra.run.trial044.stdout:(9/99): ceph-mgr-modules-core-19.2.3-675.g3c96d 4.5 MB/s | 253 kB 00:00 2026-02-17T12:01:16.996 INFO:teuthology.orchestra.run.trial044.stdout:(10/99): ceph-mgr-rook-19.2.3-675.g3c96dfb3.el9 922 kB/s | 49 kB 00:00 2026-02-17T12:01:17.051 INFO:teuthology.orchestra.run.trial044.stdout:(11/99): ceph-prometheus-alerts-19.2.3-675.g3c9 303 kB/s | 17 kB 00:00 2026-02-17T12:01:17.069 INFO:teuthology.orchestra.run.trial044.stdout:(12/99): ceph-mgr-diskprediction-local-19.2.3-6 19 MB/s | 7.4 MB 00:00 2026-02-17T12:01:17.093 INFO:teuthology.orchestra.run.trial044.stdout:(13/99): ceph-osd-19.2.3-675.g3c96dfb3.el9.x86_ 27 MB/s | 17 MB 00:00 2026-02-17T12:01:17.093 INFO:teuthology.orchestra.run.trial044.stdout:(14/99): ceph-volume-19.2.3-675.g3c96dfb3.el9.n 6.9 MB/s | 299 kB 00:00 2026-02-17T12:01:17.096 INFO:teuthology.orchestra.run.trial044.stdout:(15/99): cephadm-19.2.3-675.g3c96dfb3.el9.noarc 29 MB/s | 769 kB 00:00 2026-02-17T12:01:17.670 INFO:teuthology.orchestra.run.trial044.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 70 kB/s | 40 kB 00:00 2026-02-17T12:01:17.839 INFO:teuthology.orchestra.run.trial044.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 97 kB/s | 72 kB 00:00 2026-02-17T12:01:18.655 INFO:teuthology.orchestra.run.trial044.stdout:(18/99): libquadmath-11.5.0-14.el9.x86_64.rpm 226 kB/s | 184 kB 00:00 2026-02-17T12:01:18.727 INFO:teuthology.orchestra.run.trial044.stdout:(19/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 214 kB/s | 350 kB 00:01 2026-02-17T12:01:19.306 INFO:teuthology.orchestra.run.trial044.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 389 kB/s | 253 kB 00:00 2026-02-17T12:01:19.612 INFO:teuthology.orchestra.run.trial044.stdout:(21/99): libgfortran-11.5.0-14.el9.x86_64.rpm 409 kB/s | 794 kB 00:01 2026-02-17T12:01:19.634 INFO:teuthology.orchestra.run.trial044.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 324 kB/s | 106 kB 00:00 2026-02-17T12:01:19.880 INFO:teuthology.orchestra.run.trial044.stdout:(23/99): python3-pyparsing-2.4.7-9.el9.noarch.r 611 kB/s | 150 kB 00:00 2026-02-17T12:01:19.939 INFO:teuthology.orchestra.run.trial044.stdout:(24/99): python3-pycparser-2.20-6.el9.noarch.rp 413 kB/s | 135 kB 00:00 2026-02-17T12:01:20.127 INFO:teuthology.orchestra.run.trial044.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 514 kB/s | 126 kB 00:00 2026-02-17T12:01:20.265 INFO:teuthology.orchestra.run.trial044.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 668 kB/s | 218 kB 00:00 2026-02-17T12:01:20.453 INFO:teuthology.orchestra.run.trial044.stdout:(27/99): unzip-6.0-59.el9.x86_64.rpm 557 kB/s | 182 kB 00:00 2026-02-17T12:01:20.592 INFO:teuthology.orchestra.run.trial044.stdout:(28/99): zip-3.0-35.el9.x86_64.rpm 814 kB/s | 266 kB 00:00 2026-02-17T12:01:20.689 INFO:teuthology.orchestra.run.trial044.stdout:(29/99): python3-cryptography-36.0.1-5.el9.x86_ 650 kB/s | 1.2 MB 00:01 2026-02-17T12:01:20.716 INFO:teuthology.orchestra.run.trial044.stdout:(30/99): flexiblas-3.0.4-9.el9.x86_64.rpm 113 kB/s | 30 kB 00:00 2026-02-17T12:01:20.923 INFO:teuthology.orchestra.run.trial044.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 64 kB/s | 15 kB 00:00 2026-02-17T12:01:21.040 INFO:teuthology.orchestra.run.trial044.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 762 kB/s | 246 kB 00:00 2026-02-17T12:01:21.126 INFO:teuthology.orchestra.run.trial044.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 491 kB/s | 42 kB 00:00 2026-02-17T12:01:21.248 INFO:teuthology.orchestra.run.trial044.stdout:(34/99): lua-5.4.4-4.el9.x86_64.rpm 579 kB/s | 188 kB 00:00 2026-02-17T12:01:21.368 INFO:teuthology.orchestra.run.trial044.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 3.8 MB/s | 3.0 MB 00:00 2026-02-17T12:01:21.546 INFO:teuthology.orchestra.run.trial044.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2026-02-17T12:01:21.566 INFO:teuthology.orchestra.run.trial044.stdout:(37/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 12 MB/s | 5.3 MB 00:00 2026-02-17T12:01:21.590 INFO:teuthology.orchestra.run.trial044.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-17T12:01:21.635 INFO:teuthology.orchestra.run.trial044.stdout:(39/99): python3-devel-3.9.25-2.el9.x86_64.rpm 2.7 MB/s | 244 kB 00:00 2026-02-17T12:01:21.648 INFO:teuthology.orchestra.run.trial044.stdout:(40/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 3.0 MB/s | 249 kB 00:00 2026-02-17T12:01:21.670 INFO:teuthology.orchestra.run.trial044.stdout:(41/99): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-17T12:01:21.730 INFO:teuthology.orchestra.run.trial044.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2026-02-17T12:01:21.749 INFO:teuthology.orchestra.run.trial044.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 441 kB/s | 35 kB 00:00 2026-02-17T12:01:21.774 INFO:teuthology.orchestra.run.trial044.stdout:(44/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 8.8 MB/s | 1.2 MB 00:00 2026-02-17T12:01:21.830 INFO:teuthology.orchestra.run.trial044.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.3 MB/s | 442 kB 00:00 2026-02-17T12:01:21.855 INFO:teuthology.orchestra.run.trial044.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 951 kB/s | 77 kB 00:00 2026-02-17T12:01:21.912 INFO:teuthology.orchestra.run.trial044.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-17T12:01:21.927 INFO:teuthology.orchestra.run.trial044.stdout:(48/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 31 MB/s | 6.1 MB 00:00 2026-02-17T12:01:21.939 INFO:teuthology.orchestra.run.trial044.stdout:(49/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2026-02-17T12:01:21.993 INFO:teuthology.orchestra.run.trial044.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 3.3 MB/s | 279 kB 00:00 2026-02-17T12:01:22.007 INFO:teuthology.orchestra.run.trial044.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 672 kB/s | 54 kB 00:00 2026-02-17T12:01:22.072 INFO:teuthology.orchestra.run.trial044.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 527 kB/s | 42 kB 00:00 2026-02-17T12:01:22.087 INFO:teuthology.orchestra.run.trial044.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-17T12:01:22.152 INFO:teuthology.orchestra.run.trial044.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-17T12:01:22.318 INFO:teuthology.orchestra.run.trial044.stdout:(55/99): lua-devel-5.4.4-4.el9.x86_64.rpm 97 kB/s | 22 kB 00:00 2026-02-17T12:01:22.497 INFO:teuthology.orchestra.run.trial044.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 3.0 MB/s | 551 kB 00:00 2026-02-17T12:01:22.516 INFO:teuthology.orchestra.run.trial044.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 992 kB/s | 19 kB 00:00 2026-02-17T12:01:22.537 INFO:teuthology.orchestra.run.trial044.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 7.2 MB/s | 151 kB 00:00 2026-02-17T12:01:22.577 INFO:teuthology.orchestra.run.trial044.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 14 MB/s | 548 kB 00:00 2026-02-17T12:01:22.596 INFO:teuthology.orchestra.run.trial044.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 1.5 MB/s | 29 kB 00:00 2026-02-17T12:01:22.616 INFO:teuthology.orchestra.run.trial044.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 3.0 MB/s | 60 kB 00:00 2026-02-17T12:01:22.652 INFO:teuthology.orchestra.run.trial044.stdout:(62/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 27 MB/s | 19 MB 00:00 2026-02-17T12:01:22.653 INFO:teuthology.orchestra.run.trial044.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.2 MB/s | 43 kB 00:00 2026-02-17T12:01:22.676 INFO:teuthology.orchestra.run.trial044.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 624 kB/s | 14 kB 00:00 2026-02-17T12:01:22.696 INFO:teuthology.orchestra.run.trial044.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 8.3 MB/s | 173 kB 00:00 2026-02-17T12:01:22.718 INFO:teuthology.orchestra.run.trial044.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 17 MB/s | 358 kB 00:00 2026-02-17T12:01:22.738 INFO:teuthology.orchestra.run.trial044.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 12 MB/s | 254 kB 00:00 2026-02-17T12:01:22.756 INFO:teuthology.orchestra.run.trial044.stdout:(68/99): python3-cachetools-4.2.4-1.el9.noarch. 311 kB/s | 32 kB 00:00 2026-02-17T12:01:22.782 INFO:teuthology.orchestra.run.trial044.stdout:(69/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 47 MB/s | 2.0 MB 00:00 2026-02-17T12:01:22.803 INFO:teuthology.orchestra.run.trial044.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 6.7 MB/s | 139 kB 00:00 2026-02-17T12:01:22.814 INFO:teuthology.orchestra.run.trial044.stdout:(71/99): python3-grpcio-tools-1.46.7-10.el9.x86 2.4 MB/s | 144 kB 00:00 2026-02-17T12:01:22.823 INFO:teuthology.orchestra.run.trial044.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 2.7 MB/s | 56 kB 00:00 2026-02-17T12:01:22.834 INFO:teuthology.orchestra.run.trial044.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 536 kB/s | 11 kB 00:00 2026-02-17T12:01:22.842 INFO:teuthology.orchestra.run.trial044.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 899 kB/s | 18 kB 00:00 2026-02-17T12:01:22.854 INFO:teuthology.orchestra.run.trial044.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 1.1 MB/s | 23 kB 00:00 2026-02-17T12:01:22.862 INFO:teuthology.orchestra.run.trial044.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 1.0 MB/s | 20 kB 00:00 2026-02-17T12:01:22.874 INFO:teuthology.orchestra.run.trial044.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 978 kB/s | 19 kB 00:00 2026-02-17T12:01:22.882 INFO:teuthology.orchestra.run.trial044.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 1.3 MB/s | 26 kB 00:00 2026-02-17T12:01:22.902 INFO:teuthology.orchestra.run.trial044.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 2.3 MB/s | 46 kB 00:00 2026-02-17T12:01:22.922 INFO:teuthology.orchestra.run.trial044.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 3.9 MB/s | 79 kB 00:00 2026-02-17T12:01:22.935 INFO:teuthology.orchestra.run.trial044.stdout:(81/99): python3-kubernetes-26.1.0-3.el9.noarch 17 MB/s | 1.0 MB 00:00 2026-02-17T12:01:22.942 INFO:teuthology.orchestra.run.trial044.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 4.2 MB/s | 86 kB 00:00 2026-02-17T12:01:22.955 INFO:teuthology.orchestra.run.trial044.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 2.8 MB/s | 58 kB 00:00 2026-02-17T12:01:22.963 INFO:teuthology.orchestra.run.trial044.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 13 MB/s | 272 kB 00:00 2026-02-17T12:01:22.975 INFO:teuthology.orchestra.run.trial044.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 825 kB/s | 16 kB 00:00 2026-02-17T12:01:22.983 INFO:teuthology.orchestra.run.trial044.stdout:(86/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 4.4 MB/s | 90 kB 00:00 2026-02-17T12:01:22.995 INFO:teuthology.orchestra.run.trial044.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 1.5 MB/s | 31 kB 00:00 2026-02-17T12:01:23.000 INFO:teuthology.orchestra.run.trial044.stdout:(88/99): protobuf-compiler-3.14.0-17.el9.x86_64 1.0 MB/s | 862 kB 00:00 2026-02-17T12:01:23.003 INFO:teuthology.orchestra.run.trial044.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 9.1 MB/s | 188 kB 00:00 2026-02-17T12:01:23.015 INFO:teuthology.orchestra.run.trial044.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 2.9 MB/s | 59 kB 00:00 2026-02-17T12:01:23.023 INFO:teuthology.orchestra.run.trial044.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.8 MB/s | 36 kB 00:00 2026-02-17T12:01:23.036 INFO:teuthology.orchestra.run.trial044.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 4.2 MB/s | 86 kB 00:00 2026-02-17T12:01:23.044 INFO:teuthology.orchestra.run.trial044.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 11 MB/s | 230 kB 00:00 2026-02-17T12:01:23.056 INFO:teuthology.orchestra.run.trial044.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 4.3 MB/s | 90 kB 00:00 2026-02-17T12:01:23.066 INFO:teuthology.orchestra.run.trial044.stdout:(95/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 19 MB/s | 427 kB 00:00 2026-02-17T12:01:23.077 INFO:teuthology.orchestra.run.trial044.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.4 MB/s | 48 kB 00:00 2026-02-17T12:01:23.085 INFO:teuthology.orchestra.run.trial044.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 1.1 MB/s | 22 kB 00:00 2026-02-17T12:01:23.096 INFO:teuthology.orchestra.run.trial044.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 1.0 MB/s | 20 kB 00:00 2026-02-17T12:01:23.143 INFO:teuthology.orchestra.run.trial044.stdout:(99/99): python3-saml-1.16.0-1.el9.noarch.rpm 878 kB/s | 125 kB 00:00 2026-02-17T12:01:23.149 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:23.149 INFO:teuthology.orchestra.run.trial044.stdout:Total 13 MB/s | 95 MB 00:07 2026-02-17T12:01:23.576 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:23.613 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:23.613 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:24.033 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:24.033 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:24.481 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:24.482 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T12:01:24.514 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T12:01:24.519 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T12:01:24.540 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T12:01:24.546 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T12:01:24.579 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T12:01:24.588 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T12:01:24.593 INFO:teuthology.orchestra.run.trial044.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T12:01:24.596 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T12:01:24.600 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T12:01:24.624 INFO:teuthology.orchestra.run.trial044.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T12:01:24.636 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T12:01:24.639 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T12:01:24.645 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T12:01:24.647 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T12:01:24.658 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T12:01:24.664 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T12:01:24.688 INFO:teuthology.orchestra.run.trial044.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T12:01:24.691 INFO:teuthology.orchestra.run.trial044.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T12:01:24.698 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T12:01:24.723 INFO:teuthology.orchestra.run.trial044.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T12:01:24.774 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T12:01:24.787 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T12:01:24.789 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T12:01:24.794 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T12:01:24.799 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T12:01:24.804 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T12:01:24.807 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T12:01:24.822 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T12:01:24.843 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T12:01:24.848 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T12:01:24.853 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T12:01:24.864 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T12:01:24.873 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T12:01:24.879 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T12:01:24.899 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T12:01:24.908 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T12:01:24.963 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T12:01:24.975 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T12:01:24.991 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T12:01:24.995 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T12:01:25.003 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T12:01:25.043 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T12:01:25.371 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T12:01:25.381 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T12:01:25.385 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T12:01:25.391 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T12:01:25.393 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T12:01:25.421 INFO:teuthology.orchestra.run.trial044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T12:01:25.465 INFO:teuthology.orchestra.run.trial044.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T12:01:25.471 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T12:01:25.475 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T12:01:25.481 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T12:01:25.484 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T12:01:25.491 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T12:01:25.495 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T12:01:25.523 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T12:01:25.533 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T12:01:25.568 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T12:01:25.792 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T12:01:25.816 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T12:01:25.820 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T12:01:25.878 INFO:teuthology.orchestra.run.trial044.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T12:01:25.880 INFO:teuthology.orchestra.run.trial044.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T12:01:25.901 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T12:01:26.230 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T12:01:26.304 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T12:01:26.980 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T12:01:26.997 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T12:01:27.007 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T12:01:27.013 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T12:01:27.027 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T12:01:27.043 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T12:01:27.119 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T12:01:27.128 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T12:01:27.152 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T12:01:27.183 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T12:01:27.233 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T12:01:27.239 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T12:01:27.243 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T12:01:27.244 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T12:01:27.261 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T12:01:27.261 INFO:teuthology.orchestra.run.trial044.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T12:01:27.261 INFO:teuthology.orchestra.run.trial044.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T12:01:27.261 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:27.269 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T12:01:27.295 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T12:01:27.295 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T12:01:27.295 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:27.332 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T12:01:27.397 INFO:teuthology.orchestra.run.trial044.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T12:01:27.399 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-volume-2:19.2.3-675.g3c96dfb3.el9.noarch 85/99 2026-02-17T12:01:27.414 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-volume-2:19.2.3-675.g3c96dfb3.el9.noarch 85/99 2026-02-17T12:01:27.414 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:27.414 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T12:01:27.414 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:28.109 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-osd-2:19.2.3-675.g3c96dfb3.el9.x86_64 86/99 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-osd-2:19.2.3-675.g3c96dfb3.el9.x86_64 86/99 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T12:01:28.134 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:28.190 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 87/99 2026-02-17T12:01:28.192 INFO:teuthology.orchestra.run.trial044.stdout: Installing : cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 87/99 2026-02-17T12:01:28.197 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-675.g3c96dfb3.el9. 88/99 2026-02-17T12:01:28.217 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-675.g3c96dfb3.el9 89/99 2026-02-17T12:01:28.218 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 90/99 2026-02-17T12:01:28.711 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 90/99 2026-02-17T12:01:28.714 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-675.g3c96dfb3.el9.noar 91/99 2026-02-17T12:01:29.165 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-675.g3c96dfb3.el9.noar 91/99 2026-02-17T12:01:29.166 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-675.g3c96df 92/99 2026-02-17T12:01:29.179 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-675.g3c96df 92/99 2026-02-17T12:01:29.180 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-675.g3c96dfb3.el9.noar 93/99 2026-02-17T12:01:29.235 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-675.g3c96dfb3.el9.noar 93/99 2026-02-17T12:01:29.284 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-675.g3c96dfb3.el9.n 94/99 2026-02-17T12:01:29.285 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-2:19.2.3-675.g3c96dfb3.el9.x86_64 95/99 2026-02-17T12:01:29.306 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-2:19.2.3-675.g3c96dfb3.el9.x86_64 95/99 2026-02-17T12:01:29.306 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:29.306 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T12:01:29.306 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T12:01:29.306 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T12:01:29.307 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:29.318 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-rook-2:19.2.3-675.g3c96dfb3.el9.noarch 96/99 2026-02-17T12:01:29.534 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-675.g3c96dfb3.el9.noarch 96/99 2026-02-17T12:01:29.535 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mon-2:19.2.3-675.g3c96dfb3.el9.x86_64 97/99 2026-02-17T12:01:29.557 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mon-2:19.2.3-675.g3c96dfb3.el9.x86_64 97/99 2026-02-17T12:01:29.557 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:29.557 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T12:01:29.557 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T12:01:29.557 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T12:01:29.558 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:29.658 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mds-2:19.2.3-675.g3c96dfb3.el9.x86_64 98/99 2026-02-17T12:01:29.680 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mds-2:19.2.3-675.g3c96dfb3.el9.x86_64 98/99 2026-02-17T12:01:29.680 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:29.681 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T12:01:29.681 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T12:01:29.681 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T12:01:29.681 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:29.699 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-2:19.2.3-675.g3c96dfb3.el9.x86_64 99/99 2026-02-17T12:01:30.350 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-2:19.2.3-675.g3c96dfb3.el9.x86_64 99/99 2026-02-17T12:01:30.350 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mds-2:19.2.3-675.g3c96dfb3.el9.x86_64 2/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-2:19.2.3-675.g3c96dfb3.el9.x86_64 3/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mon-2:19.2.3-675.g3c96dfb3.el9.x86_64 4/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-osd-2:19.2.3-675.g3c96dfb3.el9.x86_64 5/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-675.g3c96dfb3.el9 6/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 7/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-675.g3c96dfb3.el9.noar 8/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-675.g3c96df 9/99 2026-02-17T12:01:30.351 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-675.g3c96dfb3.el9.noar 10/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-675.g3c96dfb3.el9.n 11/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-rook-2:19.2.3-675.g3c96dfb3.el9.noarch 12/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-675.g3c96dfb3.el9. 13/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-volume-2:19.2.3-675.g3c96dfb3.el9.noarch 14/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 15/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T12:01:30.352 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T12:01:30.353 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T12:01:30.354 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T12:01:30.355 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T12:01:30.356 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T12:01:30.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T12:01:30.358 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T12:01:30.359 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T12:01:30.359 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T12:01:30.359 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T12:01:30.411 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-grafana-dashboards-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mds-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-dashboard-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-diskprediction-local-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-k8sevents-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-modules-core-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-rook-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mon-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-prometheus-alerts-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ceph-volume-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T12:01:30.412 INFO:teuthology.orchestra.run.trial044.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T12:01:30.413 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: zip-3.0-35.el9.x86_64 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:30.414 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:30.468 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-base 2026-02-17T12:01:30.781 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:30.845 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-base-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:30.863 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:30.863 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:30.863 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:30.885 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install cephadm 2026-02-17T12:01:31.192 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:31.257 INFO:teuthology.orchestra.run.trial044.stdout:Package cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:31.274 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:31.275 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:31.275 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:31.296 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T12:01:31.605 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:31.687 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:31.687 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:31.687 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 145 k 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 145 k 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 431 k 2026-02-17T12:01:31.688 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:31.786 INFO:teuthology.orchestra.run.trial044.stdout:ceph-immutable-object-cache-19.2.3-675.g3c96dfb 1.4 MB/s | 145 kB 00:00 2026-02-17T12:01:31.787 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:31.787 INFO:teuthology.orchestra.run.trial044.stdout:Total 1.4 MB/s | 145 kB 00:00 2026-02-17T12:01:31.787 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:31.791 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:31.791 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:31.822 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:31.822 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:31.876 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:31.878 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-675.g3c96dfb3.e 1/1 2026-02-17T12:01:31.902 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-675.g3c96dfb3.e 1/1 2026-02-17T12:01:31.902 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:31.902 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T12:01:31.902 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-675.g3c96dfb3.e 1/1 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout: ceph-immutable-object-cache-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:32.190 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:32.217 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr 2026-02-17T12:01:32.528 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:32.593 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:32.610 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:32.611 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:32.611 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:32.633 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T12:01:32.944 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:33.009 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-dashboard-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:33.026 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:33.027 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:33.027 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:33.048 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T12:01:33.357 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:33.422 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:33.439 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:33.440 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:33.440 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:33.461 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-rook 2026-02-17T12:01:33.771 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:33.836 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-rook-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:33.853 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:33.854 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:33.854 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:33.875 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T12:01:34.186 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:34.251 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-cephadm-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:34.269 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:34.269 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:34.269 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:34.291 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-fuse 2026-02-17T12:01:34.599 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:34.682 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout: ceph-fuse x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 1.2 M 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:34.683 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:34.684 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 1.2 M 2026-02-17T12:01:34.684 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 3.6 M 2026-02-17T12:01:34.684 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:34.801 INFO:teuthology.orchestra.run.trial044.stdout:ceph-fuse-19.2.3-675.g3c96dfb3.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2026-02-17T12:01:34.802 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:34.802 INFO:teuthology.orchestra.run.trial044.stdout:Total 9.6 MB/s | 1.2 MB 00:00 2026-02-17T12:01:34.802 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:34.806 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:34.806 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:34.840 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:34.840 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:34.940 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:34.957 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:35.169 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout: ceph-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:35.226 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:35.253 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-volume 2026-02-17T12:01:35.567 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:35.631 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-volume-2:19.2.3-675.g3c96dfb3.el9.noarch is already installed. 2026-02-17T12:01:35.648 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:35.649 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:35.649 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:35.671 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados-devel 2026-02-17T12:01:35.985 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:36.068 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout: librados-devel x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 127 k 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:36.069 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 127 k 2026-02-17T12:01:36.070 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 456 k 2026-02-17T12:01:36.070 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:36.154 INFO:teuthology.orchestra.run.trial044.stdout:librados-devel-19.2.3-675.g3c96dfb3.el9.x86_64. 1.5 MB/s | 127 kB 00:00 2026-02-17T12:01:36.154 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:36.154 INFO:teuthology.orchestra.run.trial044.stdout:Total 1.4 MB/s | 127 kB 00:00 2026-02-17T12:01:36.155 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:36.157 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:36.157 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:36.188 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:36.188 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:36.242 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:36.259 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librados-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:36.339 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout: librados-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:36.398 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:36.427 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs2 2026-02-17T12:01:36.740 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:36.804 INFO:teuthology.orchestra.run.trial044.stdout:Package libcephfs2-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:36.821 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:36.821 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:36.821 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:36.843 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs-devel 2026-02-17T12:01:37.145 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:37.227 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:37.227 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-devel x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 34 k 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 34 k 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 153 k 2026-02-17T12:01:37.228 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:37.291 INFO:teuthology.orchestra.run.trial044.stdout:libcephfs-devel-19.2.3-675.g3c96dfb3.el9.x86_64 529 kB/s | 34 kB 00:00 2026-02-17T12:01:37.292 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:37.292 INFO:teuthology.orchestra.run.trial044.stdout:Total 519 kB/s | 34 kB 00:00 2026-02-17T12:01:37.292 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:37.294 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:37.294 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:37.296 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:37.296 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:37.317 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:37.333 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:37.392 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:37.450 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:37.450 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:37.450 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:37.450 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-devel-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:37.451 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:37.451 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:37.478 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados2 2026-02-17T12:01:37.792 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:37.856 INFO:teuthology.orchestra.run.trial044.stdout:Package librados2-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:37.874 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:37.875 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:37.875 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:37.896 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librbd1 2026-02-17T12:01:38.200 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:38.265 INFO:teuthology.orchestra.run.trial044.stdout:Package librbd1-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:38.282 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:38.282 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:38.283 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:38.304 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rados 2026-02-17T12:01:38.617 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:38.681 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rados-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:38.699 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:38.699 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:38.700 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:38.720 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rgw 2026-02-17T12:01:39.030 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:39.094 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rgw-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:39.112 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:39.113 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:39.113 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:39.132 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-cephfs 2026-02-17T12:01:39.446 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:39.510 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-cephfs-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:39.527 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:39.528 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:39.528 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:39.549 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rbd 2026-02-17T12:01:39.858 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:39.922 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 is already installed. 2026-02-17T12:01:39.940 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:39.941 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-17T12:01:39.942 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:39.961 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-fuse 2026-02-17T12:01:40.273 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout: rbd-fuse x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 85 k 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 85 k 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 227 k 2026-02-17T12:01:40.355 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:40.429 INFO:teuthology.orchestra.run.trial044.stdout:rbd-fuse-19.2.3-675.g3c96dfb3.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T12:01:40.429 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:40.429 INFO:teuthology.orchestra.run.trial044.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T12:01:40.429 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:40.432 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:40.433 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:40.464 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:40.464 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:40.515 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:40.531 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:40.611 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout: rbd-fuse-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:40.670 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:40.697 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-mirror 2026-02-17T12:01:41.012 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:44 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout: rbd-mirror x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 3.1 M 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:41.097 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:41.098 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:41.098 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:41.098 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 3.1 M 2026-02-17T12:01:41.098 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 13 M 2026-02-17T12:01:41.098 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:41.252 INFO:teuthology.orchestra.run.trial044.stdout:rbd-mirror-19.2.3-675.g3c96dfb3.el9.x86_64.rpm 20 MB/s | 3.1 MB 00:00 2026-02-17T12:01:41.252 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:41.252 INFO:teuthology.orchestra.run.trial044.stdout:Total 20 MB/s | 3.1 MB 00:00 2026-02-17T12:01:41.252 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:41.256 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:41.256 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:41.295 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:41.295 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:41.490 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:41.492 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-mirror-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:41.516 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-mirror-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:41.516 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T12:01:41.517 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T12:01:41.517 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T12:01:41.517 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T12:01:41.517 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:41.807 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-mirror-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:41.808 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:41.808 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:41.808 INFO:teuthology.orchestra.run.trial044.stdout: rbd-mirror-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:41.808 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:41.808 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:41.836 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-nbd 2026-02-17T12:01:42.147 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:45 ago on Tue 17 Feb 2026 12:00:57 PM UTC. 2026-02-17T12:01:42.231 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout: rbd-nbd x86_64 2:19.2.3-675.g3c96dfb3.el9 ceph 171 k 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-17T12:01:42.232 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-17T12:01:42.233 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-17T12:01:42.233 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:42.233 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 171 k 2026-02-17T12:01:42.233 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 490 k 2026-02-17T12:01:42.233 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-17T12:01:42.317 INFO:teuthology.orchestra.run.trial044.stdout:rbd-nbd-19.2.3-675.g3c96dfb3.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2026-02-17T12:01:42.318 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-17T12:01:42.318 INFO:teuthology.orchestra.run.trial044.stdout:Total 1.9 MB/s | 171 kB 00:00 2026-02-17T12:01:42.319 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-17T12:01:42.322 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-17T12:01:42.322 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-17T12:01:42.353 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-17T12:01:42.354 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-17T12:01:42.409 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-17T12:01:42.426 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-nbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:42.507 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-nbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:42.565 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-nbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 1/1 2026-02-17T12:01:42.565 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:42.565 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-17T12:01:42.566 INFO:teuthology.orchestra.run.trial044.stdout: rbd-nbd-2:19.2.3-675.g3c96dfb3.el9.x86_64 2026-02-17T12:01:42.566 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-17T12:01:42.566 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-17T12:01:42.597 DEBUG:teuthology.parallel:result is None 2026-02-17T12:01:42.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T12:01:42.725 DEBUG:teuthology.orchestra.run.trial044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T12:01:42.746 INFO:teuthology.orchestra.run.trial044.stdout:19.2.3-675.g3c96dfb3.el9 2026-02-17T12:01:42.746 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-675.g3c96dfb3.el9 2026-02-17T12:01:42.746 INFO:teuthology.task.install:The correct ceph version 19.2.3-675.g3c96dfb3 is installed. 2026-02-17T12:01:42.747 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T12:01:42.748 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:01:42.748 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T12:01:42.810 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T12:01:42.810 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:01:42.810 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T12:01:42.874 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T12:01:42.939 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T12:01:42.939 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:01:42.939 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T12:01:43.003 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T12:01:43.068 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T12:01:43.068 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:01:43.068 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T12:01:43.132 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T12:01:43.196 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T12:01:43.211 INFO:tasks.workunit:Pulling workunits from ref 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T12:01:43.211 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T12:01:43.211 INFO:tasks.workunit:timeout=3h 2026-02-17T12:01:43.211 INFO:tasks.workunit:cleanup=True 2026-02-17T12:01:43.212 DEBUG:teuthology.orchestra.run.trial044:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T12:01:43.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T12:01:43.254 INFO:teuthology.orchestra.run.trial044.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T12:01:43.255 DEBUG:teuthology.orchestra.run.trial044:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T12:01:43.313 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T12:01:43.313 DEBUG:teuthology.orchestra.run.trial044:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T12:01:43.368 DEBUG:teuthology.orchestra.run.trial044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T12:01:43.432 INFO:tasks.workunit.client.0.trial044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr:Note: switching to '3c96dfb3f50b3d02598a82af7104764bfdcd3919'. 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T12:02:03.676 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: git switch -c 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr:Or undo this operation with: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: git switch - 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr: 2026-02-17T12:02:03.677 INFO:tasks.workunit.client.0.trial044.stderr:HEAD is now at 3c96dfb3f50 Merge branch 'wip-ifed-non-spatial-alloc-squi' of https://github.com/ifed01/ceph into wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T12:02:03.683 DEBUG:teuthology.orchestra.run.trial044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T12:02:03.743 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-17T12:02:03.743 DEBUG:teuthology.orchestra.run.trial044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T12:02:03.798 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-17T12:02:03.799 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-17T12:02:03.799 DEBUG:teuthology.orchestra.run.trial044:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-17T12:02:03.861 INFO:tasks.workunit.client.0.trial044.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-17T12:02:03.861 INFO:tasks.workunit.client.0.trial044.stderr:++ TIMEOUT=300 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ MAX_TIMEOUT=15 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ PG_NUM=4 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ TMPDIR=/tmp 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ type xmlstarlet 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ XMLSTARLET=xmlstarlet 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:+++ uname 2026-02-17T12:02:03.862 INFO:tasks.workunit.client.0.trial044.stderr:++ '[' Linux = FreeBSD ']' 2026-02-17T12:02:03.863 INFO:tasks.workunit.client.0.trial044.stderr:++ SED=sed 2026-02-17T12:02:03.863 INFO:tasks.workunit.client.0.trial044.stderr:++ AWK=awk 2026-02-17T12:02:03.864 INFO:tasks.workunit.client.0.trial044.stderr:+++ stty -a 2026-02-17T12:02:03.864 INFO:tasks.workunit.client.0.trial044.stderr:+++ head -1 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ termwidth= 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ DIFFCOLOPTS='-y ' 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ KERNCORE=kernel.core_pattern 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ EXTRA_OPTS= 2026-02-17T12:02:03.865 INFO:tasks.workunit.client.0.trial044.stderr:++ test '' = TESTS 2026-02-17T12:02:03.868 INFO:tasks.workunit.client.0.trial044.stderr:++ uname -m 2026-02-17T12:02:03.868 INFO:tasks.workunit.client.0.trial044.stderr:+ arch=x86_64 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ case $arch in 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ main test-erasure-code-plugins 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ shift 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ shopt -s -o xtrace 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:02:03.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/test-erasure-code-plugins 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-17T12:02:03.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-17T12:02:03.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-17T12:02:03.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none ' 2026-02-17T12:02:03.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:03.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-17T12:02:03.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_warning 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:TEST_preload_no_warning 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:TEST_preload_no_warning_default 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:TEST_preload_warning' 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-17T12:02:03.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:03.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:03.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:03.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:03.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:03.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:03.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:02:03.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:02:03.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-17T12:02:03.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:02:03.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:03.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:03.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:03.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.909 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:03.909 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:03.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:02:03.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:02:03.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:02:03.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:03.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:03.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:02:03.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:03.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:02:03.983 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:03.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:03.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:03.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:02:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:02:04.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:04.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:04.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:04.143 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-17T12:02:04.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:04.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:04.149 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:04.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=68bbed77-44f2-43d4-b64c-3616daf6c691 2026-02-17T12:02:04.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 68bbed77-44f2-43d4-b64c-3616daf6c691' 2026-02-17T12:02:04.152 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 68bbed77-44f2-43d4-b64c-3616daf6c691 2026-02-17T12:02:04.152 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:04.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC8WJRpMt1XCRAAexzsxz3axZNNnNKi03rOig== 2026-02-17T12:02:04.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC8WJRpMt1XCRAAexzsxz3axZNNnNKi03rOig=="}' 2026-02-17T12:02:04.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 68bbed77-44f2-43d4-b64c-3616daf6c691 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:04.258 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:04.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:04.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC8WJRpMt1XCRAAexzsxz3axZNNnNKi03rOig== --osd-uuid 68bbed77-44f2-43d4-b64c-3616daf6c691 2026-02-17T12:02:04.284 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:04.284+0000 7fb9b8584780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:04.286 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:04.285+0000 7fb9b8584780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:04.287 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:04.286+0000 7fb9b8584780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:04.287 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:04.286+0000 7fb9b8584780 -1 bdev(0x55fde461fc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:04.288 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:04.286+0000 7fb9b8584780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:02:06.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:02:06.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:06.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:02:06.693 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:02:06.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:06.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:06.957 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:06.957+0000 7fd97f613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:06.959 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:06.958+0000 7fd97f613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:06.959 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:06.959+0000 7fd97f613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:07.128 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:07.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:07.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:07.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:07.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:08.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:08.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:08.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:08.315 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:08.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:08.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:08.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:08.543 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:08.542+0000 7fd97f613780 -1 Falling back to public interface 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:09.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:09.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:10.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:10.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:10.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:10.694 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:10.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:10.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:10.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:11.482 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:11.481+0000 7fd97f613780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:02:11.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:11.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:11.882 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:02:11.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:11.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:11.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:12.072 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1642444766,v1:127.0.0.1:6803/1642444766] [v2:127.0.0.1:6804/1642444766,v1:127.0.0.1:6805/1642444766] exists,up 68bbed77-44f2-43d4-b64c-3616daf6c691 2026-02-17T12:02:12.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:12.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:12.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:12.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-17T12:02:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fced0df0-69dc-46d0-ac33-90e2e520fb9f 2026-02-17T12:02:12.081 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 fced0df0-69dc-46d0-ac33-90e2e520fb9f 2026-02-17T12:02:12.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 fced0df0-69dc-46d0-ac33-90e2e520fb9f' 2026-02-17T12:02:12.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:12.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDEWJRpxVcoBRAAWT+RoVbKYj1Z+VYmRDAzSA== 2026-02-17T12:02:12.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDEWJRpxVcoBRAAWT+RoVbKYj1Z+VYmRDAzSA=="}' 2026-02-17T12:02:12.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fced0df0-69dc-46d0-ac33-90e2e520fb9f -i td/test-erasure-code-plugins/1/new.json 2026-02-17T12:02:12.277 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:12.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-17T12:02:12.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEWJRpxVcoBRAAWT+RoVbKYj1Z+VYmRDAzSA== --osd-uuid fced0df0-69dc-46d0-ac33-90e2e520fb9f 2026-02-17T12:02:12.301 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:12.299+0000 7fcd31523780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:12.302 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:12.300+0000 7fcd31523780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:12.303 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:12.301+0000 7fcd31523780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:12.303 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:12.302+0000 7fcd31523780 -1 bdev(0x558f778abc00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:12.303 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:12.302+0000 7fcd31523780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-17T12:02:14.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-17T12:02:14.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:14.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:02:14.417 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:02:14.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:14.680 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:02:14.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:02:14.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:14.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:14.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:14.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:14.696 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:14.694+0000 7f07fcc21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.697 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:14.695+0000 7f07fcc21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.698 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:14.696+0000 7f07fcc21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:14.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:15.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:16.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:16.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:16.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:16.056 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:16.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:16.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:16.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:16.290 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:16.288+0000 7f07fcc21780 -1 Falling back to public interface 2026-02-17T12:02:17.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:17.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:17.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:17.245 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:17.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:17.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:17.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:18.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:18.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:18.433 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:18.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:18.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:18.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:18.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:18.988 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:18.987+0000 7f07fcc21780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:02:19.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:19.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:19.629 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:02:19.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:19.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:19.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:19.817 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1128586743,v1:127.0.0.1:6811/1128586743] [v2:127.0.0.1:6812/1128586743,v1:127.0.0.1:6813/1128586743] exists,up fced0df0-69dc-46d0-ac33-90e2e520fb9f 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:19.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:19.819 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=045fadbd-4c2f-4694-a920-e6f61302cbef 2026-02-17T12:02:19.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 045fadbd-4c2f-4694-a920-e6f61302cbef' 2026-02-17T12:02:19.821 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 045fadbd-4c2f-4694-a920-e6f61302cbef 2026-02-17T12:02:19.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:19.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDLWJRpXGCWMRAADbQsdbkfcVJHJIw6lzDOXw== 2026-02-17T12:02:19.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDLWJRpXGCWMRAADbQsdbkfcVJHJIw6lzDOXw=="}' 2026-02-17T12:02:19.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 045fadbd-4c2f-4694-a920-e6f61302cbef -i td/test-erasure-code-plugins/2/new.json 2026-02-17T12:02:20.025 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:20.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-17T12:02:20.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLWJRpXGCWMRAADbQsdbkfcVJHJIw6lzDOXw== --osd-uuid 045fadbd-4c2f-4694-a920-e6f61302cbef 2026-02-17T12:02:20.050 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:20.049+0000 7f41fb60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:20.051 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:20.050+0000 7f41fb60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:20.052 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:20.051+0000 7f41fb60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:20.052 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:20.052+0000 7f41fb60d780 -1 bdev(0x55ff37e7bc00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:20.052 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:20.052+0000 7f41fb60d780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-17T12:02:22.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-17T12:02:22.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:22.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:02:22.167 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:02:22.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:22.418 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:02:22.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:02:22.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:22.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:22.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:22.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:22.434 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:22.434+0000 7f6461d23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:22.436 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:22.435+0000 7f6461d23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:22.436 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:22.436+0000 7f6461d23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:22.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:22.605 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:22.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:22.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:02:22.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:23.519 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:23.519+0000 7f6461d23780 -1 Falling back to public interface 2026-02-17T12:02:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:23.802 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:23.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:23.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:02:23.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:24.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:24.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:24.997 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:24.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:24.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:24.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:02:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:26.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:26.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:26.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:26.189 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:26.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:26.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:02:26.226 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:26.225+0000 7f6461d23780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:02:26.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1941616478,v1:127.0.0.1:6819/1941616478] [v2:127.0.0.1:6820/1941616478,v1:127.0.0.1:6821/1941616478] exists,up 045fadbd-4c2f-4694-a920-e6f61302cbef 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-17T12:02:27.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:02:27.755 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' does not exist 2026-02-17T12:02:27.762 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:02:27.762 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:02:27.998 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:02:28.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:02:29.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:29.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:29.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:29.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:29.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:29.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:29.374 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:29.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:29.561 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:02:29.561 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:02:29.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:29.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:29.561 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:29.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T12:02:29.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T12:02:29.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T12:02:29.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:29.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:29.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T12:02:29.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T12:02:29.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668' 2026-02-17T12:02:29.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:29.682 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:02:29.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T12:02:29.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T12:02:29.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668 2-55834574852' 2026-02-17T12:02:29.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:29.743 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T12:02:29.743 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:29.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:29.744 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T12:02:29.744 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:29.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T12:02:29.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T12:02:29.745 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836486 2026-02-17T12:02:29.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:29.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-17T12:02:29.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:30.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:30.934 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:31.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T12:02:31.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:31.123 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T12:02:31.123 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:31.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:31.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T12:02:31.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:31.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T12:02:31.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T12:02:31.126 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 38654705668 2026-02-17T12:02:31.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:31.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705668 2026-02-17T12:02:31.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:31.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T12:02:31.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:31.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:02:31.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T12:02:31.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:31.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T12:02:31.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T12:02:31.317 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 55834574852 2026-02-17T12:02:31.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:02:31.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-17T12:02:31.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:31.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:31.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:31.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:02:31.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:31.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:31.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:02:31.947 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:31.947 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:31.947 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:32.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:02:32.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:32.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:32.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-17T12:02:32.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.431 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:32.432 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:32.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:32.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:32.473 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:32.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:32.480 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.348+0000 7fe39012f640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.480 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.421+0000 7fe39012f640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.480 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.421+0000 7fe39012f640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-17T12:02:32.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:32.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:32.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:32.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:32.776 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:32.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:32.782 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.622+0000 7fe39012f640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.783 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.726+0000 7fe39012f640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.783 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.726+0000 7fe39012f640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:32.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-17T12:02:32.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:33.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:33.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:33.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:33.073 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:33.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:33.080 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:32.919+0000 7fe39012f640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:33.080 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.023+0000 7fe39012f640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:33.080 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.023+0000 7fe39012f640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-17T12:02:33.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-17T12:02:33.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:33.336 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:33.337 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:33.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:33.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:33.377 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:33.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:33.383 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.218+0000 7fe39012f640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.383 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.327+0000 7fe39012f640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.383 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.327+0000 7fe39012f640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-17T12:02:33.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-17T12:02:33.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:33.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:33.678 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:33.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:33.684 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.525+0000 7fe39012f640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.684 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.627+0000 7fe39012f640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.684 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.627+0000 7fe39012f640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-17T12:02:33.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:33.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:33.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-17T12:02:33.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:33.976 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:33.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:33.983 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.820+0000 7fe39012f640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.983 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.924+0000 7fe39012f640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.983 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:02:33.924+0000 7fe39012f640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-17T12:02:33.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:02:33.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:33.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:33.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:33.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:33.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:33.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:33.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:33.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:34.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:34.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:34.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:34.091 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:34.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:34.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:34.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:34.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:34.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:34.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:34.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:34.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:34.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:34.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:34.102 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:34.103 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.103 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:34.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:34.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:34.106 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:34.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:34.107 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:34.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:34.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:34.108 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:34.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:34.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:34.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:34.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:34.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:34.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:34.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:34.112 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:34.112 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.112 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:02:34.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:02:34.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:02:34.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-17T12:02:34.195 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:02:34.195 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:34.195 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:34.196 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:34.196 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.196 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.196 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:34.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-17T12:02:34.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:02:34.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:02:34.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:34.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:34.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:34.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:34.268 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:34.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:02:34.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:02:34.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:02:34.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:02:34.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:02:34.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:02:34.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:02:34.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:02:34.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:34.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:34.407 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:34.408 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.408 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:34.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:34.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:34.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-17T12:02:34.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:34.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:34.424 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:34.425 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.425 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.425 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:34.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:02:34.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:34.464 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:34.468 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-17T12:02:34.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:02:34.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:34.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1ba70009-fd26-442e-a8d1-6fcc246e22c0 2026-02-17T12:02:34.470 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 1ba70009-fd26-442e-a8d1-6fcc246e22c0 2026-02-17T12:02:34.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1ba70009-fd26-442e-a8d1-6fcc246e22c0' 2026-02-17T12:02:34.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:34.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDaWJRpDdS3HBAAhw39TD2DWw7AcePpPNZR9A== 2026-02-17T12:02:34.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDaWJRpDdS3HBAAhw39TD2DWw7AcePpPNZR9A=="}' 2026-02-17T12:02:34.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1ba70009-fd26-442e-a8d1-6fcc246e22c0 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:34.584 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:34.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:34.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQDaWJRpDdS3HBAAhw39TD2DWw7AcePpPNZR9A== --osd-uuid 1ba70009-fd26-442e-a8d1-6fcc246e22c0 2026-02-17T12:02:34.609 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:34.608+0000 7f151300d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:34.610 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:34.609+0000 7f151300d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:34.610 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:34.610+0000 7f151300d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:34.611 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:34.610+0000 7f151300d780 -1 bdev(0x5615f1b45c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:34.611 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:34.610+0000 7f151300d780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:02:37.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:02:37.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:37.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:02:37.258 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:02:37.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:37.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:37.533 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:37.532+0000 7f4c31f16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:37.534 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:37.533+0000 7f4c31f16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:37.535 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:37.534+0000 7f4c31f16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:37.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:37.706 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:37.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:37.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:37.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:38.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:38.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:38.890 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:38.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:38.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:38.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:39.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:39.128 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:39.127+0000 7f4c31f16780 -1 Falling back to public interface 2026-02-17T12:02:40.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:40.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:40.080 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:40.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:40.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:40.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:40.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:41.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:41.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:41.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:41.275 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:41.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:41.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:41.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:42.062 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:42.061+0000 7f4c31f16780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:02:42.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:42.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:42.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:42.463 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:02:42.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:42.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:42.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:43.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:43.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:43.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:02:43.654 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:02:43.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:43.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:43.836 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/286912006,v1:127.0.0.1:6803/286912006] [v2:127.0.0.1:6804/286912006,v1:127.0.0.1:6805/286912006] exists,up 1ba70009-fd26-442e-a8d1-6fcc246e22c0 2026-02-17T12:02:43.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:43.840 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:43.841 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:43.841 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:43.841 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:02:43.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:02:43.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:02:43.882 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:43.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:43.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:43.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:43.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:43.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:43.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:43.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:43.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:43.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:43.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:44.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:44.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:44.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:44.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:44.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:44.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:44.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:44.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:44.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:44.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:44.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:44.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:44.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:44.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:44.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:44.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:44.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:44.012 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:44.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:44.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:44.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:44.013 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:44.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:44.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:44.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:44.017 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:44.017 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.017 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:44.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:44.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:44.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:02:44.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:02:44.020 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.020 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:02:44.020 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:02:44.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:02:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:44.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-17T12:02:44.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:44.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:44.066 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:44.066 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.066 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.066 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:44.067 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:44.067 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:44.114 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:44.115 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:44.116 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.116 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.116 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:44.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:44.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:02:44.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:02:44.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:02:44.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:02:44.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:44.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:44.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:02:44.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:44.316 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:44.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:44.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a60180c3-5d6d-4033-919a-fecdb3ec7271 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 a60180c3-5d6d-4033-919a-fecdb3ec7271 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a60180c3-5d6d-4033-919a-fecdb3ec7271' 2026-02-17T12:02:44.323 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:44.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDkWJRpH4/tExAA2NY/Jixc5ru4U4k0tj78Pw== 2026-02-17T12:02:44.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDkWJRpH4/tExAA2NY/Jixc5ru4U4k0tj78Pw=="}' 2026-02-17T12:02:44.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a60180c3-5d6d-4033-919a-fecdb3ec7271 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:44.435 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:44.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:44.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQDkWJRpH4/tExAA2NY/Jixc5ru4U4k0tj78Pw== --osd-uuid a60180c3-5d6d-4033-919a-fecdb3ec7271 2026-02-17T12:02:44.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:44.459+0000 7fd31e433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:44.461 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:44.460+0000 7fd31e433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:44.462 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:44.461+0000 7fd31e433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:44.463 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:44.462+0000 7fd31e433780 -1 bdev(0x5642b3dcdc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:44.463 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:44.462+0000 7fd31e433780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:02:47.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:02:47.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:47.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:02:47.092 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:02:47.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:47.358 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:02:47.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:02:47.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-17T12:02:47.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:47.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:47.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:47.374 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:47.374+0000 7fde18cd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:47.376 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:47.375+0000 7fde18cd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:47.376 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:47.376+0000 7fde18cd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:47.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:02:47.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:47.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:47.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:47.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:47.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:47.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:47.545 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:47.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:47.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:47.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:48.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:48.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:48.960 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:48.960+0000 7fde18cd0780 -1 Falling back to public interface 2026-02-17T12:02:49.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:49.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:49.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:49.920 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:49.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:49.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:50.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:51.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:51.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:51.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:51.110 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:51.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:51.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:51.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:51.655 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:51.654+0000 7fde18cd0780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:02:52.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:52.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:52.303 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:02:52.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:52.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:52.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:52.489 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2508504661,v1:127.0.0.1:6803/2508504661] [v2:127.0.0.1:6804/2508504661,v1:127.0.0.1:6805/2508504661] exists,up a60180c3-5d6d-4033-919a-fecdb3ec7271 2026-02-17T12:02:52.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:52.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:52.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:52.490 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:02:52.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:02:52.535 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:52.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:02:52.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:52.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:52.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:52.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:52.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:52.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:52.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:52.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:52.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:52.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:52.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:52.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:52.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:52.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:52.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:52.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:52.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:02:52.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:52.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:52.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:02:52.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:02:52.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:02:52.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:02:52.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:02:52.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:02:52.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:52.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:02:52.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:02:52.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:02:52.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:02:52.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:02:52.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:02:52.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:02:52.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:02:52.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:02:52.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:02:52.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:02:52.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:02:52.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-17T12:02:52.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:02:52.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:02:52.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:02:52.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:02:52.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:52.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-17T12:02:52.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:02:52.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:02:52.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:52.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:52.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:02:52.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:52.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:02:52.761 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:02:52.761 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:02:52.761 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:02:52.761 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:52.762 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:02:52.763 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:02:52.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:02:52.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:02:52.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:02:52.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:02:52.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:02:52.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.903 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:52.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:52.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:02:52.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-17T12:02:52.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:02:52.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:02:52.963 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:02:52.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:52.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:52.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-17T12:02:52.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:02:52.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:02:52.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2f01099a-6a70-4859-8374-5738c2630a31 2026-02-17T12:02:52.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2f01099a-6a70-4859-8374-5738c2630a31' 2026-02-17T12:02:52.973 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 2f01099a-6a70-4859-8374-5738c2630a31 2026-02-17T12:02:52.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:02:52.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDsWJRpbKlsOhAA3Ug8Kak9TZoEeg18t7vESg== 2026-02-17T12:02:52.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDsWJRpbKlsOhAA3Ug8Kak9TZoEeg18t7vESg=="}' 2026-02-17T12:02:52.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2f01099a-6a70-4859-8374-5738c2630a31 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:53.083 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:53.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:02:53.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQDsWJRpbKlsOhAA3Ug8Kak9TZoEeg18t7vESg== --osd-uuid 2f01099a-6a70-4859-8374-5738c2630a31 2026-02-17T12:02:53.109 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:53.109+0000 7f88ad53f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.111 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:53.110+0000 7f88ad53f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.111 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:53.111+0000 7f88ad53f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.112 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:53.111+0000 7f88ad53f780 -1 bdev(0x5621e8ce3c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:02:53.112 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:53.111+0000 7f88ad53f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:02:55.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:02:55.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:02:55.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:02:55.747 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:02:55.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:02:56.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:02:56.020 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:56.019+0000 7f02b8e1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:56.022 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:56.020+0000 7f02b8e1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:56.022 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:56.021+0000 7f02b8e1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:56.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:56.190 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:02:56.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:56.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:56.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:57.376 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:57.376 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:57.376 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:02:57.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:57.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:57.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:57.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:57.621 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:02:57.619+0000 7f02b8e1f780 -1 Falling back to public interface 2026-02-17T12:02:58.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:58.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:58.566 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:02:58.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:58.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:58.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:58.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:59.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:59.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:59.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:59.756 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:02:59.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:59.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:59.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:00.555 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:00.554+0000 7f02b8e1f780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:00.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:00.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:00.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:00.954 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:00.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:00.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:01.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:02.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:02.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:02.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:02.148 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:02.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:02.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:02.337 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3293587865,v1:127.0.0.1:6803/3293587865] [v2:127.0.0.1:6804/3293587865,v1:127.0.0.1:6805/3293587865] exists,up 2f01099a-6a70-4859-8374-5738c2630a31 2026-02-17T12:03:02.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:03:02.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:02.383 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:02.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:02.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:02.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:02.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:02.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:02.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:02.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:02.498 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:02.498 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:02.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:02.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:02.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:02.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:02.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:02.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:02.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:02.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:02.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:02.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:02.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:02.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:02.509 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:02.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:02.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:02.511 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:02.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:02.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:02.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:02.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:02.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:02.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:02.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:02.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:02.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:02.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:02.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:02.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:02.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:02.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:02.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:02.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:02.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:02.520 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:02.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:02.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:02.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:02.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:02.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:02.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-17T12:03:02.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:02.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:02.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:02.568 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:02.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:02.614 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:02.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:02.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:02.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:02.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:02.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:02.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:02.753 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:02.753 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.754 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.754 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:02.754 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:02.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:02.811 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:02.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:02.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:02.821 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:02.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=63370bd0-524b-4eb7-803a-4074b71f5691 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 63370bd0-524b-4eb7-803a-4074b71f5691' 2026-02-17T12:03:02.823 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 63370bd0-524b-4eb7-803a-4074b71f5691 2026-02-17T12:03:02.824 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:02.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD2WJRpyHB/MRAAgBme+t5/0EyuB0QK3tgQ7A== 2026-02-17T12:03:02.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD2WJRpyHB/MRAAgBme+t5/0EyuB0QK3tgQ7A=="}' 2026-02-17T12:03:02.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 63370bd0-524b-4eb7-803a-4074b71f5691 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:02.934 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:02.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:02.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQD2WJRpyHB/MRAAgBme+t5/0EyuB0QK3tgQ7A== --osd-uuid 63370bd0-524b-4eb7-803a-4074b71f5691 2026-02-17T12:03:02.958 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:02.958+0000 7feb1674f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:02.960 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:02.959+0000 7feb1674f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:02.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:02.960+0000 7feb1674f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:02.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:02.961+0000 7feb1674f780 -1 bdev(0x56091907bc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:02.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:02.961+0000 7feb1674f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:05.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:05.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:05.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:05.604 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:05.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:05.856 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:05.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:05.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-17T12:03:05.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:05.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:05.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:05.871 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:05.871+0000 7fda9722c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:05.873 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:05.872+0000 7fda9722c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:05.874 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:05.873+0000 7fda9722c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:06.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:06.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:07.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:07.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:07.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:07.231 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:07.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:07.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:07.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:07.464 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:07.463+0000 7fda9722c780 -1 Falling back to public interface 2026-02-17T12:03:08.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:08.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:08.419 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:08.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:08.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:08.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:08.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:09.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:09.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:09.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:09.610 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:09.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:09.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:09.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:10.395 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:10.395+0000 7fda9722c780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:10.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:10.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:11.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:11.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:11.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:11.990 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:11.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:11.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:12.179 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2058620175,v1:127.0.0.1:6803/2058620175] [v2:127.0.0.1:6804/2058620175,v1:127.0.0.1:6805/2058620175] exists,up 63370bd0-524b-4eb7-803a-4074b71f5691 2026-02-17T12:03:12.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:12.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-17T12:03:12.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:12.226 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:12.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:12.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:12.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:12.235 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:12.235 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:12.235 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:12.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:12.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:12.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:12.342 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:12.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:12.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:12.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:12.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:12.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:12.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:12.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:12.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:12.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:12.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:12.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:12.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:12.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:12.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:12.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-17T12:03:12.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-17T12:03:12.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:12.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:12.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:12.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:12.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:12.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:12.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:12.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:12.357 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:12.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:12.358 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:12.358 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:12.358 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:12.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:12.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:12.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:12.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:12.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:12.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:12.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:12.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:12.363 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:12.363 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.363 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:12.364 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:12.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:12.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:12.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:12.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:12.384 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:12.384 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.385 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:12.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:03:12.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:12.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:12.410 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:12.411 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:12.411 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.411 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:12.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:12.412 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:12.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:12.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:12.455 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:12.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:12.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-17T12:03:12.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:12.538 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:12.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:12.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:12.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:12.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:12.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:12.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:12.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:03:12.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:12.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:12.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d0a1f82f-527c-471d-b889-60501ff4f096 2026-02-17T12:03:12.664 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 d0a1f82f-527c-471d-b889-60501ff4f096 2026-02-17T12:03:12.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 d0a1f82f-527c-471d-b889-60501ff4f096' 2026-02-17T12:03:12.665 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:12.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAAWZRp4ngeKBAAMH40cr93q1F2nsxOGDiqcg== 2026-02-17T12:03:12.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAAWZRp4ngeKBAAMH40cr93q1F2nsxOGDiqcg=="}' 2026-02-17T12:03:12.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d0a1f82f-527c-471d-b889-60501ff4f096 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:12.774 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:12.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:12.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAAWZRp4ngeKBAAMH40cr93q1F2nsxOGDiqcg== --osd-uuid d0a1f82f-527c-471d-b889-60501ff4f096 2026-02-17T12:03:12.797 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:12.796+0000 7faf7e82a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:12.798 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:12.797+0000 7faf7e82a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:12.799 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:12.798+0000 7faf7e82a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:12.799 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:12.799+0000 7faf7e82a780 -1 bdev(0x556352379c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:12.800 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:12.799+0000 7faf7e82a780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:15.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:15.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:15.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:15.205 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:15.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:15.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:15.458 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:15.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:03:15.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:15.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:15.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:15.473 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:15.472+0000 7fe1f24d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:15.474 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:15.474+0000 7fe1f24d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:15.475 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:15.474+0000 7fe1f24d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:15.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:15.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:15.644 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:15.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:15.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:15.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:16.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:16.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:16.839 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:16.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:16.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:16.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:17.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:17.067 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:17.066+0000 7fe1f24d9780 -1 Falling back to public interface 2026-02-17T12:03:18.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:18.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:18.033 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:18.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:18.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:18.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:18.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:19.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:19.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:19.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:19.222 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:19.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:19.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:19.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:20.006 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:20.005+0000 7fe1f24d9780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:20.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:20.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:20.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:20.411 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:20.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:21.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:21.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:21.601 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:21.795 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2357352926,v1:127.0.0.1:6803/2357352926] [v2:127.0.0.1:6804/2357352926,v1:127.0.0.1:6805/2357352926] exists,up d0a1f82f-527c-471d-b889-60501ff4f096 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:21.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-17T12:03:21.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:21.840 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:21.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:21.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:21.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:21.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:21.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:21.954 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:21.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:21.955 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:21.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:21.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:21.957 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:21.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:21.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:21.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:21.959 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:21.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:21.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:21.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:21.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:21.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:21.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:21.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:21.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:21.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-17T12:03:21.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-17T12:03:21.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:21.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:21.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:21.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:21.968 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:21.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:21.969 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:21.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:21.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:21.970 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:21.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:21.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:21.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:21.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:21.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:21.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:21.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:21.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:21.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:21.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:21.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:21.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:21.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:21.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:21.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:21.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:21.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:21.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:21.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:21.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:21.998 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:21.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:22.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:22.024 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:22.025 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.025 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.025 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:22.026 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:22.026 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:22.026 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:22.071 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:22.072 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:22.073 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:22.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:22.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:22.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:22.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:22.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:22.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:22.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:22.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:22.279 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:22.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:22.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:22.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:22.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:22.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a5438d20-cad6-4adf-b8fa-3b7fd66f1020 2026-02-17T12:03:22.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a5438d20-cad6-4adf-b8fa-3b7fd66f1020' 2026-02-17T12:03:22.290 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 a5438d20-cad6-4adf-b8fa-3b7fd66f1020 2026-02-17T12:03:22.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:22.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAKWZRpT4+3ERAAVCSDQBSigJ7gKM6Xb4PbVg== 2026-02-17T12:03:22.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAKWZRpT4+3ERAAVCSDQBSigJ7gKM6Xb4PbVg=="}' 2026-02-17T12:03:22.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a5438d20-cad6-4adf-b8fa-3b7fd66f1020 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:22.398 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:22.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:22.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQAKWZRpT4+3ERAAVCSDQBSigJ7gKM6Xb4PbVg== --osd-uuid a5438d20-cad6-4adf-b8fa-3b7fd66f1020 2026-02-17T12:03:22.423 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:22.422+0000 7fb36b01c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:22.424 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:22.423+0000 7fb36b01c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:22.425 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:22.424+0000 7fb36b01c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:22.425 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:22.424+0000 7fb36b01c780 -1 bdev(0x559ce5675c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:22.426 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:22.425+0000 7fb36b01c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:25.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:25.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:25.055 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:25.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:25.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:25.318 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:25.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:25.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-17T12:03:25.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:25.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:25.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:25.335 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:25.334+0000 7f27f9afe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:25.336 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:25.335+0000 7f27f9afe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:25.337 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:25.336+0000 7f27f9afe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:25.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:25.509 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:25.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:25.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:25.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:26.669 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:26.668+0000 7f27f9afe780 -1 Falling back to public interface 2026-02-17T12:03:26.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:26.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:26.697 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:26.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:26.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:26.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:26.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:27.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:27.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:27.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:27.885 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:27.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:27.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:28.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:29.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:29.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:29.076 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:29.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:29.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:29.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:29.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:29.387 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:29.386+0000 7f27f9afe780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:30.267 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:30.267 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:30.268 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:30.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:30.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:30.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:30.456 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3053230148,v1:127.0.0.1:6803/3053230148] [v2:127.0.0.1:6804/3053230148,v1:127.0.0.1:6805/3053230148] exists,up a5438d20-cad6-4adf-b8fa-3b7fd66f1020 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.458 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:30.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:30.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:30.504 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:30.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:30.511 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:22.014+0000 7f94f8c0ed80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:30.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:30.512 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:26.669+0000 7f27f9afe780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:30.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:30.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:30.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:30.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:30.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:30.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:30.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:30.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:30.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:30.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:30.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:30.621 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:30.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:30.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:30.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:30.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:30.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:30.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:30.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:30.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:30.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:30.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:30.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:30.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:30.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:30.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:30.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:03:30.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:30.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:30.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:30.634 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:30.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:30.635 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:30.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:30.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:30.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:30.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:30.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:30.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:30.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:30.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:30.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:30.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:30.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:30.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:30.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:30.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:30.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:30.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:30.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:30.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:30.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:30.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:30.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:30.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:30.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:30.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:30.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:30.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:30.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:30.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:30.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:30.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:30.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:30.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:30.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:30.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:30.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:30.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:30.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:30.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:30.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:30.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:30.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:30.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:30.878 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:30.878 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.878 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.878 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:30.878 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:30.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:30.895 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:30.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:30.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:30.935 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:30.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:30.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:30.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=51e4bb97-0e8a-4f60-aa74-4b1eff04a875 2026-02-17T12:03:30.945 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 51e4bb97-0e8a-4f60-aa74-4b1eff04a875 2026-02-17T12:03:30.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 51e4bb97-0e8a-4f60-aa74-4b1eff04a875' 2026-02-17T12:03:30.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:30.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQASWZRpOV7LOBAAl+Gly3ANfnUFdhTgGc7Udw== 2026-02-17T12:03:30.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQASWZRpOV7LOBAAl+Gly3ANfnUFdhTgGc7Udw=="}' 2026-02-17T12:03:30.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 51e4bb97-0e8a-4f60-aa74-4b1eff04a875 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:31.054 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:31.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:31.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQASWZRpOV7LOBAAl+Gly3ANfnUFdhTgGc7Udw== --osd-uuid 51e4bb97-0e8a-4f60-aa74-4b1eff04a875 2026-02-17T12:03:31.078 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:31.076+0000 7f632abd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:31.079 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:31.077+0000 7f632abd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:31.080 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:31.078+0000 7f632abd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:31.080 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:31.078+0000 7f632abd4780 -1 bdev(0x5596c6d29c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:31.081 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:31.078+0000 7f632abd4780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:33.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:33.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:33.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:33.728 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:33.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:33.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:33.978 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:33.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-17T12:03:33.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:33.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:33.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:33.994 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:33.993+0000 7f2464d53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:33.996 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:33.994+0000 7f2464d53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:33.996 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:33.995+0000 7f2464d53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:34.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:34.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:34.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:34.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:34.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:34.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:34.166 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:34.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:34.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:34.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:34.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:35.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:35.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:35.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:35.353 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:35.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:35.578 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:35.576+0000 7f2464d53780 -1 Falling back to public interface 2026-02-17T12:03:36.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:36.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:36.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:36.538 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:36.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:36.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:36.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:37.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:37.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:37.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:37.725 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:37.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:37.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:37.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:38.516 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:38.515+0000 7f2464d53780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:38.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:38.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:38.914 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:38.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:38.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:38.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:39.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:40.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:40.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:40.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:40.109 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:40.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:40.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1563277889,v1:127.0.0.1:6803/1563277889] [v2:127.0.0.1:6804/1563277889,v1:127.0.0.1:6805/1563277889] exists,up 51e4bb97-0e8a-4f60-aa74-4b1eff04a875 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:40.298 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.299 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:40.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:40.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:40.345 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:40.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:40.350 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:30.678+0000 7f258760ed80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:40.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:35.577+0000 7f2464d53780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:40.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:40.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:40.460 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:40.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:40.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:40.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:40.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:40.463 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:40.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:40.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:40.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:40.464 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:40.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:40.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:40.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:40.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:40.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:40.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:40.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:40.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:40.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:40.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:40.473 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:40.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:40.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:40.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:40.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:40.475 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:40.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:40.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:40.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:40.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:40.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:40.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:40.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:40.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:40.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:40.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:40.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:40.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:40.481 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:40.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:40.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:40.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:40.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:40.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:40.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:40.529 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:40.529 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.530 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:40.531 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:40.531 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:40.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:40.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:40.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:40.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:40.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:40.576 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:40.576 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:40.577 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:40.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:40.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:40.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:40.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:40.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:40.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:40.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:40.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:40.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:40.717 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:40.717 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.717 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.717 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:40.717 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:40.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:40.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:03:40.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:40.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:40.734 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:40.734 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.735 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:40.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:40.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:40.774 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:40.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:40.783 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:40.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9fddd62c-b966-4e1c-a131-b259b01f8342 2026-02-17T12:03:40.785 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 9fddd62c-b966-4e1c-a131-b259b01f8342 2026-02-17T12:03:40.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9fddd62c-b966-4e1c-a131-b259b01f8342' 2026-02-17T12:03:40.786 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:40.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAcWZRpoDtELxAArtlOe2V8OcacF3xLs+4ZKg== 2026-02-17T12:03:40.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAcWZRpoDtELxAArtlOe2V8OcacF3xLs+4ZKg=="}' 2026-02-17T12:03:40.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9fddd62c-b966-4e1c-a131-b259b01f8342 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:40.895 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:40.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:40.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQAcWZRpoDtELxAArtlOe2V8OcacF3xLs+4ZKg== --osd-uuid 9fddd62c-b966-4e1c-a131-b259b01f8342 2026-02-17T12:03:40.919 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:40.918+0000 7f6a9aad9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.920 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:40.920+0000 7f6a9aad9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.921 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:40.921+0000 7f6a9aad9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.922 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:40.921+0000 7f6a9aad9780 -1 bdev(0x55e640925c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:40.922 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:40.921+0000 7f6a9aad9780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:43.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:43.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:43.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:43.549 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:43.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:43.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:43.820 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:43.819+0000 7f7ab1e3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:43.822 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:43.821+0000 7f7ab1e3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:43.822 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:43.822+0000 7f7ab1e3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:43.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:43.993 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:43.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:43.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:44.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:45.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:45.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:45.178 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:45.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:45.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:45.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:45.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:45.405 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:45.404+0000 7f7ab1e3f780 -1 Falling back to public interface 2026-02-17T12:03:46.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:46.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:46.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:46.367 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:46.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:46.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:46.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:47.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:47.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:47.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:47.557 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:47.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:47.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:47.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:48.341 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:48.340+0000 7f7ab1e3f780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:48.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:48.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:48.745 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:48.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:48.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:48.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:48.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:49.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:49.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:49.932 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:49.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:49.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:49.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:50.125 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/989713057,v1:127.0.0.1:6803/989713057] [v2:127.0.0.1:6804/989713057,v1:127.0.0.1:6805/989713057] exists,up 9fddd62c-b966-4e1c-a131-b259b01f8342 2026-02-17T12:03:50.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:50.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:50.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:50.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:50.171 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:50.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:50.177 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:40.518+0000 7f01301cbd80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:50.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:50.178 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:45.406+0000 7f7ab1e3f780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:50.179 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:50.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:50.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:50.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:50.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:50.285 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:50.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:50.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:50.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:50.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:50.289 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:50.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:50.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:50.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:50.291 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:50.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:50.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:50.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:50.295 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:50.296 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.296 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:50.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:50.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:50.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:50.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:50.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:50.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:50.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:50.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:50.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:50.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:50.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:50.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:50.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:50.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:50.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:50.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:50.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:50.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:50.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:50.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:50.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:50.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.355 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:50.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:50.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:50.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:50.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:50.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:50.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:50.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:50.401 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:50.401 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:50.401 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:50.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:50.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:03:50.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:50.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:50.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:50.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:03:50.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:50.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:50.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:50.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:03:50.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:50.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:50.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:03:50.603 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:50.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:50.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:50.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:03:50.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:50.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=aa11f707-6f93-41f6-82a5-84bec9c7ae2c 2026-02-17T12:03:50.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 aa11f707-6f93-41f6-82a5-84bec9c7ae2c' 2026-02-17T12:03:50.615 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 aa11f707-6f93-41f6-82a5-84bec9c7ae2c 2026-02-17T12:03:50.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:50.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAmWZRpINEWJRAAz9u2VfO6KrJzZkKKry0ksQ== 2026-02-17T12:03:50.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAmWZRpINEWJRAAz9u2VfO6KrJzZkKKry0ksQ=="}' 2026-02-17T12:03:50.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new aa11f707-6f93-41f6-82a5-84bec9c7ae2c -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:50.725 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:50.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:03:50.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQAmWZRpINEWJRAAz9u2VfO6KrJzZkKKry0ksQ== --osd-uuid aa11f707-6f93-41f6-82a5-84bec9c7ae2c 2026-02-17T12:03:50.748 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:50.748+0000 7fe4d7c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:50.750 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:50.749+0000 7fe4d7c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:50.751 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:50.750+0000 7fe4d7c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:50.751 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:50.750+0000 7fe4d7c10780 -1 bdev(0x561caaf5fc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:50.751 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:50.750+0000 7fe4d7c10780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:03:53.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:03:53.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:53.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:53.144 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:03:53.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:53.396 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:03:53.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:53.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-17T12:03:53.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:53.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:53.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:53.412 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:53.411+0000 7f15f4015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:53.413 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:53.413+0000 7f15f4015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:53.414 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:53.414+0000 7f15f4015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:53.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:53.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:53.585 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:03:53.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:53.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:53.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:54.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:54.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:54.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:54.773 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:03:54.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:54.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:54.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:54.992 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:54.991+0000 7f15f4015780 -1 Falling back to public interface 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:55.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:56.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:57.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:57.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:57.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:57.151 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:03:57.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:57.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:57.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:57.936 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:03:57.935+0000 7f15f4015780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:58.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:58.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:58.341 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:03:58.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:58.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:58.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:58.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:59.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:59.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:59.532 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:03:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:59.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1739202165,v1:127.0.0.1:6803/1739202165] [v2:127.0.0.1:6804/1739202165,v1:127.0.0.1:6805/1739202165] exists,up aa11f707-6f93-41f6-82a5-84bec9c7ae2c 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:59.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:03:59.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:03:59.768 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:03:59.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:03:59.773 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:50.344+0000 7f7ea89ddd80 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:03:59.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:03:59.774 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:54.993+0000 7f15f4015780 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:59.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:59.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:59.882 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:59.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:59.883 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:59.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:59.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:59.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:59.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:59.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:59.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:59.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:59.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:59.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:59.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:59.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:59.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:59.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:59.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:59.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:59.895 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:59.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:59.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:59.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:59.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:59.897 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:59.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:59.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:59.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:59.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:59.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:59.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:59.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:03:59.901 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:59.901 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.901 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:03:59.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:59.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:59.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:03:59.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:59.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:03:59.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:03:59.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-17T12:03:59.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:59.952 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:59.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:03:59.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:59.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:59.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:59.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:59.997 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:03:59.998 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:03:59.998 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:59.998 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:04:00.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:04:00.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:00.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:04:00.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:04:00.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:04:00.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:04:00.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:04:00.140 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:00.141 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:00.142 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:00.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:00.158 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:04:00.158 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:04:00.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:04:00.198 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:04:00.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-17T12:04:00.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:00.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:00.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:00.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:04:00.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:00.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4 2026-02-17T12:04:00.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4' 2026-02-17T12:04:00.210 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4 2026-02-17T12:04:00.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:00.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAwWZRpAzXqDBAAiAexM4q9NWv35DOvE07mlQ== 2026-02-17T12:04:00.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAwWZRpAzXqDBAAiAexM4q9NWv35DOvE07mlQ=="}' 2026-02-17T12:04:00.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4 -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:04:00.315 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:00.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:04:00.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQAwWZRpAzXqDBAAiAexM4q9NWv35DOvE07mlQ== --osd-uuid cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4 2026-02-17T12:04:00.339 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:00.338+0000 7fdf0b83f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:00.340 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:00.339+0000 7fdf0b83f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:00.341 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:00.340+0000 7fdf0b83f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:00.342 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:00.341+0000 7fdf0b83f780 -1 bdev(0x559433ef3c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:00.342 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:00.341+0000 7fdf0b83f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:04:03.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:04:03.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:03.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:04:03.227 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:04:03.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:03.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:03.496 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:03.494+0000 7efebe920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:03.497 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:03.495+0000 7efebe920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:03.498 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:03.496+0000 7efebe920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:03.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:03.671 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:03.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:03.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:03.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:03.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:04.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:04.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:04.860 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:04.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:04.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:04.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:05.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:05.080 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:05.077+0000 7efebe920780 -1 Falling back to public interface 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:06.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:06.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:07.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:07.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:07.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:07.240 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:07.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:07.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:07.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:08.021 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:08.019+0000 7efebe920780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:04:08.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:08.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:08.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:08.427 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:08.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:08.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:08.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:09.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:09.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:09.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:09.620 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:04:09.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:09.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:09.808 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2941066199,v1:127.0.0.1:6803/2941066199] [v2:127.0.0.1:6804/2941066199,v1:127.0.0.1:6805/2941066199] exists,up cfa27705-47b7-45d9-9bd1-e2bfb9a33fc4 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:09.810 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:09.811 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:09.811 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:04:09.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:04:09.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:04:09.853 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:04:09.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:04:09.859 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:03:59.942+0000 7f300f617d80 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:04:09.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:04:09.860 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:04:05.079+0000 7efebe920780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:09.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:09.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:09.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:09.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:09.968 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:09.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:09.969 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:09.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:09.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:09.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:09.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:09.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:09.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:09.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:09.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:09.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:09.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:04:09.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:09.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:09.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:09.977 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code-plugins 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:09.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:04:09.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:09.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:09.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:09.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:09.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:09.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:09.980 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:09.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:09.981 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:09.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:09.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:09.983 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:09.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:09.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:09.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:09.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:09.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:09.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:09.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:04:09.987 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:09.987 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:09.987 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:09.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:04:09.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:09.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:09.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code-plugins 2026-02-17T12:04:09.989 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:04:09.989 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:09.989 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:09.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.37453 2026-02-17T12:04:09.990 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-17T12:04:09.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:10.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:04:10.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:10.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:10.011 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:10.011 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.012 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.012 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:10.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:10.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:10.038 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:10.039 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:10.039 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.039 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:04:10.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:10.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get fsid 2026-02-17T12:04:10.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:04:10.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:10.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:10.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:04:10.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:10.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.37453/ceph-mon.a.asok config get mon_host 2026-02-17T12:04:10.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-17T12:04:10.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:04:10.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:04:10.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:10.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:10.230 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:10.230 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.230 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:10.231 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:10.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:10.248 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-17T12:04:10.248 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:10.248 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-mon.a.asok 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:04:10.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-mon.a.asok log flush 2026-02-17T12:04:10.289 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:10.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:10.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:10.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:10.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:10.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:10.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:10.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:10.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-17T12:04:10.300 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:10.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7d8bd74f-0611-4be2-b5c3-8467a2914f6f 2026-02-17T12:04:10.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 7d8bd74f-0611-4be2-b5c3-8467a2914f6f' 2026-02-17T12:04:10.300 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 7d8bd74f-0611-4be2-b5c3-8467a2914f6f 2026-02-17T12:04:10.300 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:10.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA6WZRphfZdEhAAPi/5n7PSVUrZNgRwShG10Q== 2026-02-17T12:04:10.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA6WZRphfZdEhAAPi/5n7PSVUrZNgRwShG10Q=="}' 2026-02-17T12:04:10.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7d8bd74f-0611-4be2-b5c3-8467a2914f6f -i td/test-erasure-code-plugins/0/new.json 2026-02-17T12:04:10.411 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:10.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-17T12:04:10.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQA6WZRphfZdEhAAPi/5n7PSVUrZNgRwShG10Q== --osd-uuid 7d8bd74f-0611-4be2-b5c3-8467a2914f6f 2026-02-17T12:04:10.435 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:10.434+0000 7f2d1a80c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:10.437 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:10.435+0000 7f2d1a80c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:10.438 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:10.436+0000 7f2d1a80c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:10.438 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:10.436+0000 7f2d1a80c780 -1 bdev(0x55d3b4d7fc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:10.438 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:10.436+0000 7f2d1a80c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-17T12:04:13.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-17T12:04:13.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:13.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:04:13.074 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:04:13.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:13.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:04:13.324 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:04:13.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5e2a8fef-8c9b-4a36-ac52-baaa5db39425 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37453/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-17T12:04:13.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:13.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:13.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:13.339 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:13.339+0000 7f386aa1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:13.341 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:13.340+0000 7f386aa1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:13.342 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:13.341+0000 7f386aa1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:13.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:13.513 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:13.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:13.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:13.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:13.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:14.432 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:14.431+0000 7f386aa1e780 -1 Falling back to public interface 2026-02-17T12:04:14.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:14.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:14.708 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:14.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:14.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:14.709 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:14.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:15.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:15.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:15.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:15.896 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:15.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:15.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:16.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:17.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:17.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:17.400 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:17.399+0000 7f386aa1e780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:04:18.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:18.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:18.276 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:18.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:18.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:18.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:18.472 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3124092817,v1:127.0.0.1:6803/3124092817] [v2:127.0.0.1:6804/3124092817,v1:127.0.0.1:6805/3124092817] exists,up 7d8bd74f-0611-4be2-b5c3-8467a2914f6f 2026-02-17T12:04:18.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:18.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:18.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.37453/ceph-osd.0.asok 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-17T12:04:18.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37453/ceph-osd.0.asok log flush 2026-02-17T12:04:18.520 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:04:18.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-17T12:04:18.526 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:04:10.026+0000 7fe112235d80 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:04:18.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-17T12:04:18.527 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:04:14.432+0000 7f386aa1e780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-17T12:04:18.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-17T12:04:18.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:18.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:18.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:04:18.529 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:18.529 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:18.529 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:18.529 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:18.529 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:18.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:18.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:18.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:18.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:18.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:18.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:18.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:18.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:18.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:18.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:18.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:18.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:04:18.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:18.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:18.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:04:18.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:18.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:18.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-17T12:04:18.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/test-erasure-code-plugins 0 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code-plugins 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:18.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:18.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:18.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:18.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:18.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:18.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:18.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:18.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:18.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:04:18.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code-plugins 2026-02-17T12:04:18.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:18.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.37453 2026-02-17T12:04:18.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.37453 2026-02-17T12:04:18.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:18.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:18.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:04:18.657 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:04:18.657 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:04:18.723 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-17T12:04:18.723 DEBUG:teuthology.orchestra.run.trial044:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-17T12:04:18.785 INFO:tasks.workunit.client.0.trial044.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:04:18.790 INFO:tasks.workunit.client.0.trial044.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:04:18.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/test-erasure-code 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none ' 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-code 2026-02-17T12:04:18.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-code 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code KILL 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:18.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:18.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:18.796 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:18.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:18.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:18.796 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:18.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:18.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:18.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code 2026-02-17T12:04:18.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:18.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:18.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.63880 2026-02-17T12:04:18.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:18.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:18.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-code 2026-02-17T12:04:18.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:04:18.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:18.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.63880 2026-02-17T12:04:18.801 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-17T12:04:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-code 2026-02-17T12:04:18.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:04:18.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:04:18.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:04:18.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-code/a 2026-02-17T12:04:18.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:18.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:18.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:04:18.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:04:18.846 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:04:18.846 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:18.847 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.63880/ceph-mon.a.asok 2026-02-17T12:04:18.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:18.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.63880/ceph-mon.a.asok config get fsid 2026-02-17T12:04:18.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:04:18.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:18.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:18.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:18.891 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.63880/ceph-mon.a.asok 2026-02-17T12:04:18.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:18.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.63880/ceph-mon.a.asok config get mon_host 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-code 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-code/x 2026-02-17T12:04:18.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:04:19.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:04:19.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:19.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:19.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-17T12:04:19.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:19.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.63880/ceph-mon.a.asok 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-17T12:04:19.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.63880/ceph-mon.a.asok log flush 2026-02-17T12:04:19.098 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:04:19.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-17T12:04:19.103 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:04:18.837+0000 7fdf781dbd80 0 load: jerasure load: lrc 2026-02-17T12:04:19.103 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-17T12:04:19.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/0 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:19.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:19.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/0 2026-02-17T12:04:19.110 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:19.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d9518292-546f-4e35-8ea4-9815556c75d8 2026-02-17T12:04:19.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 d9518292-546f-4e35-8ea4-9815556c75d8' 2026-02-17T12:04:19.111 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 d9518292-546f-4e35-8ea4-9815556c75d8 2026-02-17T12:04:19.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:19.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBDWZRpCXEYBxAAlhL4xufrYNR0WE874Iy4IQ== 2026-02-17T12:04:19.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBDWZRpCXEYBxAAlhL4xufrYNR0WE874Iy4IQ=="}' 2026-02-17T12:04:19.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d9518292-546f-4e35-8ea4-9815556c75d8 -i td/test-erasure-code/0/new.json 2026-02-17T12:04:19.223 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:19.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/0/new.json 2026-02-17T12:04:19.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDWZRpCXEYBxAAlhL4xufrYNR0WE874Iy4IQ== --osd-uuid d9518292-546f-4e35-8ea4-9815556c75d8 2026-02-17T12:04:19.249 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:19.248+0000 7fddb6ba3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:19.250 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:19.249+0000 7fddb6ba3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:19.251 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:19.250+0000 7fddb6ba3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:19.251 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:19.250+0000 7fddb6ba3780 -1 bdev(0x55d875019c00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:19.251 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:19.250+0000 7fddb6ba3780 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-17T12:04:22.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-17T12:04:22.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:22.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:04:22.146 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:04:22.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:22.398 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:04:22.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:04:22.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:22.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:22.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:22.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:22.415 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:22.414+0000 7f72cb8c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:22.416 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:22.415+0000 7f72cb8c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:22.417 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:22.416+0000 7f72cb8c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:22.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:04:22.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:22.587 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:22.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:22.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:22.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:23.743 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:23.742+0000 7f72cb8c6780 -1 Falling back to public interface 2026-02-17T12:04:23.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:23.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:23.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:23.774 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:23.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:23.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:23.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:24.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:24.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:24.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:24.966 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:24.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:24.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:25.160 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:26.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:26.435 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:26.434+0000 7f72cb8c6780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:04:27.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:27.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:27.351 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:27.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:27.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:27.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:27.538 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2613681735,v1:127.0.0.1:6803/2613681735] [v2:127.0.0.1:6804/2613681735,v1:127.0.0.1:6805/2613681735] exists,up d9518292-546f-4e35-8ea4-9815556c75d8 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:04:27.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/1 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:27.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:27.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/1 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5fe03ec4-0e06-4734-8190-16cb1a682150 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 5fe03ec4-0e06-4734-8190-16cb1a682150' 2026-02-17T12:04:27.545 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 5fe03ec4-0e06-4734-8190-16cb1a682150 2026-02-17T12:04:27.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:27.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBLWZRpO1LlIBAAeBxlZImvsIc7PpmqLJHJJQ== 2026-02-17T12:04:27.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBLWZRpO1LlIBAAeBxlZImvsIc7PpmqLJHJJQ=="}' 2026-02-17T12:04:27.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5fe03ec4-0e06-4734-8190-16cb1a682150 -i td/test-erasure-code/1/new.json 2026-02-17T12:04:27.746 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:27.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/1/new.json 2026-02-17T12:04:27.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLWZRpO1LlIBAAeBxlZImvsIc7PpmqLJHJJQ== --osd-uuid 5fe03ec4-0e06-4734-8190-16cb1a682150 2026-02-17T12:04:27.771 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:27.770+0000 7f2472e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:27.772 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:27.771+0000 7f2472e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:27.773 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:27.772+0000 7f2472e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:27.773 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:27.772+0000 7f2472e0d780 -1 bdev(0x5641669adc00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:27.774 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:27.773+0000 7f2472e0d780 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-17T12:04:29.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-17T12:04:29.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:29.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:04:29.907 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:04:29.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:30.158 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:04:30.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:04:30.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:30.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:30.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:30.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:30.174 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:30.173+0000 7f18bc497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:30.175 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:30.174+0000 7f18bc497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:30.176 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:30.175+0000 7f18bc497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:30.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:04:30.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:30.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:04:30.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:30.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:30.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:30.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:30.351 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:30.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:30.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:31.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:31.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:31.540 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:31.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:31.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:31.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:31.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:31.771 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:31.770+0000 7f18bc497780 -1 Falling back to public interface 2026-02-17T12:04:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:32.730 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:32.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:33.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:33.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:33.919 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:33.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:33.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:33.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:34.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:34.734 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:34.733+0000 7f18bc497780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:04:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:35.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:35.105 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:35.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:35.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:35.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:36.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:36.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:36.313 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:04:36.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:36.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:36.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:36.499 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3756447569,v1:127.0.0.1:6811/3756447569] [v2:127.0.0.1:6812/3756447569,v1:127.0.0.1:6813/3756447569] exists,up 5fe03ec4-0e06-4734-8190-16cb1a682150 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/2 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:36.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:36.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/2 2026-02-17T12:04:36.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:36.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1688e36c-ddd7-4572-b6ee-cc43fb004133 2026-02-17T12:04:36.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 1688e36c-ddd7-4572-b6ee-cc43fb004133' 2026-02-17T12:04:36.503 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 1688e36c-ddd7-4572-b6ee-cc43fb004133 2026-02-17T12:04:36.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBUWZRp84+jHhAAGu40FS+jRDkW8EAQLyNzyg== 2026-02-17T12:04:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBUWZRp84+jHhAAGu40FS+jRDkW8EAQLyNzyg=="}' 2026-02-17T12:04:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1688e36c-ddd7-4572-b6ee-cc43fb004133 -i td/test-erasure-code/2/new.json 2026-02-17T12:04:36.705 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:36.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/2/new.json 2026-02-17T12:04:36.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUWZRp84+jHhAAGu40FS+jRDkW8EAQLyNzyg== --osd-uuid 1688e36c-ddd7-4572-b6ee-cc43fb004133 2026-02-17T12:04:36.729 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:36.728+0000 7f8b8c60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:36.731 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:36.730+0000 7f8b8c60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:36.732 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:36.731+0000 7f8b8c60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:36.732 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:36.731+0000 7f8b8c60c780 -1 bdev(0x55eb0ef03c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:36.732 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:36.731+0000 7f8b8c60c780 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-17T12:04:38.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-17T12:04:38.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:38.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:04:38.846 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:04:38.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:39.103 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:04:39.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:04:39.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:39.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:39.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:39.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:39.120 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:39.119+0000 7f14dae11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:39.121 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:39.120+0000 7f14dae11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:39.122 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:39.121+0000 7f14dae11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:39.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:39.290 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:39.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:39.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:39.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:40.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:40.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:40.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:40.479 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:40.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:40.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:40.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:40.702 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:40.702+0000 7f14dae11780 -1 Falling back to public interface 2026-02-17T12:04:41.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:41.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:41.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:41.671 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:41.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:41.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:41.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:42.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:42.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:42.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:42.865 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:42.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:42.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:43.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:43.656 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:43.654+0000 7f14dae11780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:04:44.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:44.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:44.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:44.057 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:44.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:44.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:44.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:45.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:45.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:45.250 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:04:45.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:45.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:45.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:45.439 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/948423205,v1:127.0.0.1:6819/948423205] [v2:127.0.0.1:6820/948423205,v1:127.0.0.1:6821/948423205] exists,up 1688e36c-ddd7-4572-b6ee-cc43fb004133 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:04:45.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/3 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:45.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:45.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/3 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:45.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=13436238-57c1-4815-b8b9-922dc46198c9 2026-02-17T12:04:45.447 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 13436238-57c1-4815-b8b9-922dc46198c9 2026-02-17T12:04:45.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 13436238-57c1-4815-b8b9-922dc46198c9' 2026-02-17T12:04:45.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:45.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBdWZRp/Q8HGxAA1U9IvVhjiNI9dqw04aLwDg== 2026-02-17T12:04:45.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBdWZRp/Q8HGxAA1U9IvVhjiNI9dqw04aLwDg=="}' 2026-02-17T12:04:45.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 13436238-57c1-4815-b8b9-922dc46198c9 -i td/test-erasure-code/3/new.json 2026-02-17T12:04:45.649 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:45.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/3/new.json 2026-02-17T12:04:45.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBdWZRp/Q8HGxAA1U9IvVhjiNI9dqw04aLwDg== --osd-uuid 13436238-57c1-4815-b8b9-922dc46198c9 2026-02-17T12:04:45.674 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:45.672+0000 7ffa33c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:45.675 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:45.673+0000 7ffa33c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:45.676 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:45.674+0000 7ffa33c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:45.676 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:45.675+0000 7ffa33c0c780 -1 bdev(0x55be21c5bc00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:45.677 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:45.675+0000 7ffa33c0c780 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-17T12:04:47.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-17T12:04:47.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:47.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:04:47.798 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:04:47.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:48.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:48.069 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:48.067+0000 7f499a013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:48.070 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:48.068+0000 7f499a013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:48.071 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:48.069+0000 7f499a013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:48.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:04:48.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:48.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:04:48.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:48.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:48.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:48.242 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:48.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:48.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:48.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:48.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:49.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:49.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:49.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:49.433 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:49.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:49.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:49.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:49.661 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:49.659+0000 7f499a013780 -1 Falling back to public interface 2026-02-17T12:04:50.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:50.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:50.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:50.628 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:50.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:50.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:50.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:51.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:51.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:51.821 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:04:51.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:51.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:51.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:52.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:52.607 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:52.606+0000 7f499a013780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:04:53.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:53.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:53.009 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:53.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:53.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:53.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:53.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:54.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:54.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:54.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:54.203 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:04:54.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:54.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:04:54.391 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3606272046,v1:127.0.0.1:6827/3606272046] [v2:127.0.0.1:6828/3606272046,v1:127.0.0.1:6829/3606272046] exists,up 13436238-57c1-4815-b8b9-922dc46198c9 2026-02-17T12:04:54.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:54.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/4 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:54.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:54.397 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:54.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/4 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a2aa66af-2730-46f7-ad88-def5c61c149f 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 a2aa66af-2730-46f7-ad88-def5c61c149f' 2026-02-17T12:04:54.399 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 a2aa66af-2730-46f7-ad88-def5c61c149f 2026-02-17T12:04:54.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:54.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBmWZRpVy87GBAAxbCfcdd589+Jgz7CWS4Ccw== 2026-02-17T12:04:54.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBmWZRpVy87GBAAxbCfcdd589+Jgz7CWS4Ccw=="}' 2026-02-17T12:04:54.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a2aa66af-2730-46f7-ad88-def5c61c149f -i td/test-erasure-code/4/new.json 2026-02-17T12:04:54.601 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:04:54.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/4/new.json 2026-02-17T12:04:54.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBmWZRpVy87GBAAxbCfcdd589+Jgz7CWS4Ccw== --osd-uuid a2aa66af-2730-46f7-ad88-def5c61c149f 2026-02-17T12:04:54.628 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:54.627+0000 7f44dd00c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:54.629 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:54.628+0000 7f44dd00c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:54.630 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:54.629+0000 7f44dd00c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:54.630 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:54.629+0000 7f44dd00c780 -1 bdev(0x55b5d55c3c00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:54.630 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:54.629+0000 7f44dd00c780 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-17T12:04:56.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-17T12:04:56.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:56.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:04:56.757 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:04:56.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:57.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:04:57.009 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:04:57.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:57.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:57.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:57.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:57.024 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:57.023+0000 7fe456813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:57.026 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:57.025+0000 7fe456813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:57.026 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:57.025+0000 7fe456813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:57.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:57.197 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:04:57.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:57.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:57.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:04:57.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:58.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:58.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:58.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:58.382 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:04:58.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:58.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:04:58.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:58.614 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:04:58.613+0000 7fe456813780 -1 Falling back to public interface 2026-02-17T12:04:59.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:59.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:59.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:59.574 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:04:59.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:59.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:04:59.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:00.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:00.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:00.767 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:00.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:00.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:00.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:05:00.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:01.550 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:01.549+0000 7fe456813780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:05:01.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:01.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:01.960 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:01.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:01.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:01.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:05:02.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:03.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:03.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:03.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:03.154 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:03.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:03.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:05:03.342 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2566440711,v1:127.0.0.1:6835/2566440711] [v2:127.0.0.1:6836/2566440711,v1:127.0.0.1:6837/2566440711] exists,up a2aa66af-2730-46f7-ad88-def5c61c149f 2026-02-17T12:05:03.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:03.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/5 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:03.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:03.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/5 2026-02-17T12:05:03.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:03.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4619773a-ed76-4068-a2ec-d89c6166d738 2026-02-17T12:05:03.346 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 4619773a-ed76-4068-a2ec-d89c6166d738 2026-02-17T12:05:03.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 4619773a-ed76-4068-a2ec-d89c6166d738' 2026-02-17T12:05:03.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:03.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBvWZRpKzVXFRAAygJVIqkIjQrFzr1dvGUu5g== 2026-02-17T12:05:03.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBvWZRpKzVXFRAAygJVIqkIjQrFzr1dvGUu5g=="}' 2026-02-17T12:05:03.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4619773a-ed76-4068-a2ec-d89c6166d738 -i td/test-erasure-code/5/new.json 2026-02-17T12:05:03.552 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:03.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/5/new.json 2026-02-17T12:05:03.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvWZRpKzVXFRAAygJVIqkIjQrFzr1dvGUu5g== --osd-uuid 4619773a-ed76-4068-a2ec-d89c6166d738 2026-02-17T12:05:03.578 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:03.577+0000 7f1d7b795780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:03.579 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:03.579+0000 7f1d7b795780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:03.580 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:03.579+0000 7f1d7b795780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:03.581 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:03.580+0000 7f1d7b795780 -1 bdev(0x563db1997c00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:03.581 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:03.580+0000 7f1d7b795780 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-17T12:05:05.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-17T12:05:05.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:05.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:05:05.696 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:05:05.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:05.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:05:05.950 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:05:05.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:05.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:05.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:05.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:05.966 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:05.965+0000 7f3b6fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:05.967 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:05.967+0000 7f3b6fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:05.968 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:05.968+0000 7f3b6fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:06.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:06.137 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:06.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:06.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:06.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:06.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:07.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:07.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:07.325 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:07.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:07.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:07.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:07.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:07.549 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:07.548+0000 7f3b6fe0f780 -1 Falling back to public interface 2026-02-17T12:05:08.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:08.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:08.514 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:08.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:08.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:08.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:08.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:09.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:09.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:09.702 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:09.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:09.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:09.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:09.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:10.515 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:10.514+0000 7f3b6fe0f780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:05:10.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:10.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:10.889 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:10.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:10.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:10.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:11.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:12.080 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:12.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:12.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:12.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:05:12.269 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1126853027,v1:127.0.0.1:6843/1126853027] [v2:127.0.0.1:6844/1126853027,v1:127.0.0.1:6845/1126853027] exists,up 4619773a-ed76-4068-a2ec-d89c6166d738 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/6 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-17T12:05:12.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:12.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:12.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:12.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:12.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:12.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:12.273 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:12.273 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:12.273 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:12.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/6 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6' 2026-02-17T12:05:12.275 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 2026-02-17T12:05:12.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:12.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB4WZRpaBz0EBAADwY6nqLpI6JrSKJtFldSow== 2026-02-17T12:05:12.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB4WZRpaBz0EBAADwY6nqLpI6JrSKJtFldSow=="}' 2026-02-17T12:05:12.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 -i td/test-erasure-code/6/new.json 2026-02-17T12:05:12.480 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:05:12.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/6/new.json 2026-02-17T12:05:12.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB4WZRpaBz0EBAADwY6nqLpI6JrSKJtFldSow== --osd-uuid 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 2026-02-17T12:05:12.505 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:12.504+0000 7fdb54537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:12.506 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:12.506+0000 7fdb54537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:12.507 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:12.506+0000 7fdb54537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:12.508 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:12.507+0000 7fdb54537780 -1 bdev(0x55a5c95bfc00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:12.508 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:12.507+0000 7fdb54537780 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-17T12:05:14.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-17T12:05:14.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:14.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:05:14.637 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:05:14.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:14.894 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:05:14.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:05:14.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:14.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:14.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:14.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:14.911 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:14.910+0000 7f0a9c097780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:14.912 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:14.911+0000 7f0a9c097780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:14.913 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:14.912+0000 7f0a9c097780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:15.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:15.082 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:15.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:15.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:15.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:15.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:16.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:16.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:16.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:16.270 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:16.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:16.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:16.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:16.499 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:16.498+0000 7f0a9c097780 -1 Falling back to public interface 2026-02-17T12:05:17.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:17.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:17.457 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:17.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:17.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:17.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:17.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:18.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:18.649 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:18.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:19.447 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:19.446+0000 7f0a9c097780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:05:19.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:19.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:19.836 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:19.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:19.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:19.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:20.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:21.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:21.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:21.036 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:21.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:21.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:21.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:05:21.223 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 32 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/4174900722,v1:127.0.0.1:6851/4174900722] [v2:127.0.0.1:6852/4174900722,v1:127.0.0.1:6853/4174900722] exists,up 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 2026-02-17T12:05:21.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/7 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:21.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:21.228 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/7 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=81eed05e-e044-47ff-8ef0-de805c3e56af 2026-02-17T12:05:21.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 81eed05e-e044-47ff-8ef0-de805c3e56af' 2026-02-17T12:05:21.231 INFO:tasks.workunit.client.0.trial044.stdout:add osd7 81eed05e-e044-47ff-8ef0-de805c3e56af 2026-02-17T12:05:21.231 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:21.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCBWZRpoSIjDhAAaEd+85cFpnSX/ybfBSKZUg== 2026-02-17T12:05:21.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCBWZRpoSIjDhAAaEd+85cFpnSX/ybfBSKZUg=="}' 2026-02-17T12:05:21.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 81eed05e-e044-47ff-8ef0-de805c3e56af -i td/test-erasure-code/7/new.json 2026-02-17T12:05:21.433 INFO:tasks.workunit.client.0.trial044.stdout:7 2026-02-17T12:05:21.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/7/new.json 2026-02-17T12:05:21.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBWZRpoSIjDhAAaEd+85cFpnSX/ybfBSKZUg== --osd-uuid 81eed05e-e044-47ff-8ef0-de805c3e56af 2026-02-17T12:05:21.458 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:21.457+0000 7f11d9a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:21.459+0000 7f11d9a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.461 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:21.460+0000 7f11d9a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.461 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:21.460+0000 7f11d9a14780 -1 bdev(0x55adfc417c00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:21.461 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:21.460+0000 7f11d9a14780 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-17T12:05:23.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-17T12:05:23.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:23.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T12:05:23.581 INFO:tasks.workunit.client.0.trial044.stdout:adding osd7 key to auth repository 2026-02-17T12:05:23.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stdout:start osd.7 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:23.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:23.853 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:23.851+0000 7f35276ba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:23.854 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:23.852+0000 7f35276ba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:23.855 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:23.853+0000 7f35276ba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:24.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T12:05:24.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:24.032 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:24.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:24.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:24.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:25.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:25.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:25.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:25.224 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:25.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:25.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:25.450 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:25.448+0000 7f35276ba780 -1 Falling back to public interface 2026-02-17T12:05:26.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:26.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:26.411 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:26.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:26.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:26.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:26.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:27.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:27.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:27.603 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:27.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:28.391 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:28.389+0000 7f35276ba780 -1 osd.7 0 log_to_monitors true 2026-02-17T12:05:28.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:28.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:28.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:28.793 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:28.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:28.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:28.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:29.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:29.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:29.988 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:29.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:29.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:29.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:05:30.176 INFO:tasks.workunit.client.0.trial044.stdout:osd.7 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1259969137,v1:127.0.0.1:6859/1259969137] [v2:127.0.0.1:6860/1259969137,v1:127.0.0.1:6861/1259969137] exists,up 81eed05e-e044-47ff-8ef0-de805c3e56af 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/8 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-17T12:05:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:30.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/8 2026-02-17T12:05:30.179 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:30.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8f7cba3c-7211-4b66-ad64-9837d94db4ad 2026-02-17T12:05:30.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 8f7cba3c-7211-4b66-ad64-9837d94db4ad' 2026-02-17T12:05:30.181 INFO:tasks.workunit.client.0.trial044.stdout:add osd8 8f7cba3c-7211-4b66-ad64-9837d94db4ad 2026-02-17T12:05:30.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:30.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCKWZRpVM93CxAAB2Q+iR7kcxHNtNhLMOIjwQ== 2026-02-17T12:05:30.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCKWZRpVM93CxAAB2Q+iR7kcxHNtNhLMOIjwQ=="}' 2026-02-17T12:05:30.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8f7cba3c-7211-4b66-ad64-9837d94db4ad -i td/test-erasure-code/8/new.json 2026-02-17T12:05:30.388 INFO:tasks.workunit.client.0.trial044.stdout:8 2026-02-17T12:05:30.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/8/new.json 2026-02-17T12:05:30.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKWZRpVM93CxAAB2Q+iR7kcxHNtNhLMOIjwQ== --osd-uuid 8f7cba3c-7211-4b66-ad64-9837d94db4ad 2026-02-17T12:05:30.413 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:30.412+0000 7f4d50d37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:30.415 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:30.414+0000 7f4d50d37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:30.415 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:30.415+0000 7f4d50d37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:30.416 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:30.415+0000 7f4d50d37780 -1 bdev(0x5605ddfcfc00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:30.416 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:30.415+0000 7f4d50d37780 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-17T12:05:32.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-17T12:05:32.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:32.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T12:05:32.528 INFO:tasks.workunit.client.0.trial044.stdout:adding osd8 key to auth repository 2026-02-17T12:05:32.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:32.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T12:05:32.794 INFO:tasks.workunit.client.0.trial044.stdout:start osd.8 2026-02-17T12:05:32.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:32.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:32.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:32.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:32.810 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:32.809+0000 7fa574e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:32.811 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:32.810+0000 7fa574e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:32.812 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:32.811+0000 7fa574e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:32.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:32.985 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:32.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:32.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:32.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:33.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:34.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:34.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:34.180 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:34.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:34.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:34.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:34.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:34.401 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:34.400+0000 7fa574e0c780 -1 Falling back to public interface 2026-02-17T12:05:35.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:35.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:35.373 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:35.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:35.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:35.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:35.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:36.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:36.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:36.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:36.565 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:36.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:36.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:36.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:37.365 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:37.363+0000 7fa574e0c780 -1 osd.8 0 log_to_monitors true 2026-02-17T12:05:37.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:37.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:37.753 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:37.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:37.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:37.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:37.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:38.948 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:38.948 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:38.948 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:38.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:38.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:38.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:05:39.138 INFO:tasks.workunit.client.0.trial044.stdout:osd.8 up in weight 1 up_from 41 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1578961363,v1:127.0.0.1:6867/1578961363] [v2:127.0.0.1:6868/1578961363,v1:127.0.0.1:6869/1578961363] exists,up 8f7cba3c-7211-4b66-ad64-9837d94db4ad 2026-02-17T12:05:39.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:39.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:39.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:39.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=9 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/9 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:39.139 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:39.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/9 2026-02-17T12:05:39.141 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:39.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c7f42f79-0655-43ec-b04a-ee7db982e022 2026-02-17T12:05:39.142 INFO:tasks.workunit.client.0.trial044.stdout:add osd9 c7f42f79-0655-43ec-b04a-ee7db982e022 2026-02-17T12:05:39.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd9 c7f42f79-0655-43ec-b04a-ee7db982e022' 2026-02-17T12:05:39.142 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:39.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCTWZRpc0cmCRAAWAGtqxPhmDWuXBk2wT/0fw== 2026-02-17T12:05:39.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCTWZRpc0cmCRAAWAGtqxPhmDWuXBk2wT/0fw=="}' 2026-02-17T12:05:39.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c7f42f79-0655-43ec-b04a-ee7db982e022 -i td/test-erasure-code/9/new.json 2026-02-17T12:05:39.347 INFO:tasks.workunit.client.0.trial044.stdout:9 2026-02-17T12:05:39.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/9/new.json 2026-02-17T12:05:39.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 9 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCTWZRpc0cmCRAAWAGtqxPhmDWuXBk2wT/0fw== --osd-uuid c7f42f79-0655-43ec-b04a-ee7db982e022 2026-02-17T12:05:39.372 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:39.372+0000 7f88ab49d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:39.374 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:39.373+0000 7f88ab49d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:39.375 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:39.374+0000 7f88ab49d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:39.375 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:39.374+0000 7f88ab49d780 -1 bdev(0x562b614c1c00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:39.375 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:39.374+0000 7f88ab49d780 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-17T12:05:41.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-17T12:05:41.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:41.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd9 key to auth repository 2026-02-17T12:05:41.746 INFO:tasks.workunit.client.0.trial044.stdout:adding osd9 key to auth repository 2026-02-17T12:05:41.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:41.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.9 2026-02-17T12:05:41.998 INFO:tasks.workunit.client.0.trial044.stdout:start osd.9 2026-02-17T12:05:41.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 9 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:41.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:41.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:41.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:42.015 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:42.014+0000 7f9992ad0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:42.017 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:42.016+0000 7f9992ad0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:42.017 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:42.017+0000 7f9992ad0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:42.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 9 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:42.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:43.376 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:43.377 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:43.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:43.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:43.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:43.601 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:43.600+0000 7f9992ad0780 -1 Falling back to public interface 2026-02-17T12:05:44.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:44.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:44.567 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:44.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:44.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:44.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:44.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:45.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:46.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:46.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:46.960 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:46.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:46.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:46.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:47.075 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:47.074+0000 7f9992ad0780 -1 osd.9 0 log_to_monitors true 2026-02-17T12:05:47.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:05:48.336 INFO:tasks.workunit.client.0.trial044.stdout:osd.9 up in weight 1 up_from 45 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/273161259,v1:127.0.0.1:6875/273161259] [v2:127.0.0.1:6876/273161259,v1:127.0.0.1:6877/273161259] exists,up c7f42f79-0655-43ec-b04a-ee7db982e022 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-code 2026-02-17T12:05:48.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=10 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-code/10 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:48.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-code/10 2026-02-17T12:05:48.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:48.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b2ec5acb-aad2-4503-9954-c706f87f79f5 2026-02-17T12:05:48.341 INFO:tasks.workunit.client.0.trial044.stdout:add osd10 b2ec5acb-aad2-4503-9954-c706f87f79f5 2026-02-17T12:05:48.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd10 b2ec5acb-aad2-4503-9954-c706f87f79f5' 2026-02-17T12:05:48.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:48.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCcWZRpXLX3FBAAfWmSeeUfr/Zs+hdS+NJCHQ== 2026-02-17T12:05:48.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCcWZRpXLX3FBAAfWmSeeUfr/Zs+hdS+NJCHQ=="}' 2026-02-17T12:05:48.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b2ec5acb-aad2-4503-9954-c706f87f79f5 -i td/test-erasure-code/10/new.json 2026-02-17T12:05:48.545 INFO:tasks.workunit.client.0.trial044.stdout:10 2026-02-17T12:05:48.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-code/10/new.json 2026-02-17T12:05:48.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 10 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCcWZRpXLX3FBAAfWmSeeUfr/Zs+hdS+NJCHQ== --osd-uuid b2ec5acb-aad2-4503-9954-c706f87f79f5 2026-02-17T12:05:48.572 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:48.571+0000 7f678aee7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:48.574 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:48.573+0000 7f678aee7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:48.575 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:48.574+0000 7f678aee7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:48.575 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:48.574+0000 7f678aee7780 -1 bdev(0x559c6382dc00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:48.575 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:48.574+0000 7f678aee7780 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-17T12:05:50.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-17T12:05:50.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:50.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd10 key to auth repository 2026-02-17T12:05:50.688 INFO:tasks.workunit.client.0.trial044.stdout:adding osd10 key to auth repository 2026-02-17T12:05:50.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:50.944 INFO:tasks.workunit.client.0.trial044.stdout:start osd.10 2026-02-17T12:05:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.10 2026-02-17T12:05:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 10 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:50.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:50.960+0000 7f4fc5984780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:50.962 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:50.961+0000 7f4fc5984780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:50.963 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:50.962+0000 7f4fc5984780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:51.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 10 2026-02-17T12:05:51.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:51.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=10 2026-02-17T12:05:51.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:51.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:51.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:51.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:51.134 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:05:51.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:51.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:51.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:52.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:52.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:52.798 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:52.797+0000 7f4fc5984780 -1 Falling back to public interface 2026-02-17T12:05:53.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:53.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:53.515 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:05:53.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:53.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:53.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:53.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:54.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:54.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:55.751 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:05:55.750+0000 7f4fc5984780 -1 osd.10 0 log_to_monitors true 2026-02-17T12:05:55.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:55.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:55.894 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:05:55.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:55.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:55.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:56.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:57.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:57.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:57.087 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:05:57.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:57.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:57.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.10 up' 2026-02-17T12:05:57.274 INFO:tasks.workunit.client.0.trial044.stdout:osd.10 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/780599570,v1:127.0.0.1:6883/780599570] [v2:127.0.0.1:6884/780599570,v1:127.0.0.1:6885/780599570] exists,up b2ec5acb-aad2-4503-9954-c706f87f79f5 2026-02-17T12:05:57.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:57.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:57.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:57.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-17T12:05:57.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:05:57.461 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' does not exist 2026-02-17T12:05:57.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:05:57.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:05:57.707 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:05:57.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:05:58.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:05:59.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-17T12:05:59.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:05:59.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:05:59.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:05:59.000 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:05:59.001 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:05:59.001 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:05:59.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:05:59.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:05:59.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:05:59.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:05:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:05:59.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:05:59.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.283 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:05:59.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T12:05:59.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T12:05:59.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T12:05:59.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:05:59.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672980 2026-02-17T12:05:59.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672980 2026-02-17T12:05:59.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980' 2026-02-17T12:05:59.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:05:59.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T12:05:59.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T12:05:59.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161' 2026-02-17T12:05:59.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.471 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:05:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411344 2026-02-17T12:05:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411344 2026-02-17T12:05:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344' 2026-02-17T12:05:59.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.532 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:05:59.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247823 2026-02-17T12:05:59.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247823 2026-02-17T12:05:59.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823' 2026-02-17T12:05:59.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:05:59.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117004 2026-02-17T12:05:59.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117004 2026-02-17T12:05:59.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004' 2026-02-17T12:05:59.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:05:59.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953482 2026-02-17T12:05:59.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953482 2026-02-17T12:05:59.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004 6-137438953482' 2026-02-17T12:05:59.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.718 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:05:59.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822666 2026-02-17T12:05:59.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822666 2026-02-17T12:05:59.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004 6-137438953482 7-154618822666' 2026-02-17T12:05:59.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:05:59.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659143 2026-02-17T12:05:59.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659143 2026-02-17T12:05:59.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004 6-137438953482 7-154618822666 8-176093659143' 2026-02-17T12:05:59.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.841 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:05:59.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528326 2026-02-17T12:05:59.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528326 2026-02-17T12:05:59.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004 6-137438953482 7-154618822666 8-176093659143 9-193273528326' 2026-02-17T12:05:59.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:59.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:05:59.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397507 2026-02-17T12:05:59.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397507 2026-02-17T12:05:59.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-60129542161 3-77309411344 4-98784247823 5-115964117004 6-137438953482 7-154618822666 8-176093659143 9-193273528326 10-210453397507' 2026-02-17T12:05:59.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:59.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T12:05:59.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:59.967 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:05:59.967 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T12:05:59.968 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:59.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T12:05:59.969 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836501 2026-02-17T12:05:59.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T12:05:59.969 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836501 2026-02-17T12:06:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:01.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:01.157 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:01.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T12:06:01.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:01.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672980 2026-02-17T12:06:01.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:01.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:01.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672980 2026-02-17T12:06:01.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:01.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672980 2026-02-17T12:06:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672980' 2026-02-17T12:06:01.346 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 42949672980 2026-02-17T12:06:01.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:01.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672981 -lt 42949672980 2026-02-17T12:06:01.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:01.531 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T12:06:01.531 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:01.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:01.532 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T12:06:01.533 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:01.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T12:06:01.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T12:06:01.534 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 60129542161 2026-02-17T12:06:01.535 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:01.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542162 -lt 60129542161 2026-02-17T12:06:01.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:01.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411344 2026-02-17T12:06:01.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:01.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:01.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411344 2026-02-17T12:06:01.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:01.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411344 2026-02-17T12:06:01.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411344' 2026-02-17T12:06:01.728 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 77309411344 2026-02-17T12:06:01.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:01.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411344 -lt 77309411344 2026-02-17T12:06:01.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:01.921 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247823 2026-02-17T12:06:01.921 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:01.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:01.922 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247823 2026-02-17T12:06:01.922 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:01.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247823 2026-02-17T12:06:01.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247823' 2026-02-17T12:06:01.924 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 98784247823 2026-02-17T12:06:01.924 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:02.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247823 -lt 98784247823 2026-02-17T12:06:02.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:02.113 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117004 2026-02-17T12:06:02.113 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:02.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:06:02.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117004 2026-02-17T12:06:02.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:02.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117004 2026-02-17T12:06:02.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117004' 2026-02-17T12:06:02.116 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 115964117004 2026-02-17T12:06:02.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:06:02.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117004 -lt 115964117004 2026-02-17T12:06:02.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:02.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-137438953482 2026-02-17T12:06:02.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:02.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:06:02.307 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-137438953482 2026-02-17T12:06:02.307 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:02.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953482 2026-02-17T12:06:02.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 137438953482' 2026-02-17T12:06:02.309 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 137438953482 2026-02-17T12:06:02.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:06:02.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953483 -lt 137438953482 2026-02-17T12:06:02.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:02.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822666 2026-02-17T12:06:02.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:02.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:06:02.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822666 2026-02-17T12:06:02.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:02.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822666 2026-02-17T12:06:02.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822666' 2026-02-17T12:06:02.505 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 154618822666 2026-02-17T12:06:02.505 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:06:02.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822666 -lt 154618822666 2026-02-17T12:06:02.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:02.695 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659143 2026-02-17T12:06:02.695 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:02.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:06:02.696 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659143 2026-02-17T12:06:02.696 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:02.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659143 2026-02-17T12:06:02.697 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 176093659143 2026-02-17T12:06:02.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659143' 2026-02-17T12:06:02.698 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:06:02.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659143 -lt 176093659143 2026-02-17T12:06:02.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:02.886 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528326 2026-02-17T12:06:02.886 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:02.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:06:02.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528326 2026-02-17T12:06:02.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:02.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528326 2026-02-17T12:06:02.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528326' 2026-02-17T12:06:02.889 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.9 seq 193273528326 2026-02-17T12:06:02.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:06:03.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528326 -lt 193273528326 2026-02-17T12:06:03.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:03.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397507 2026-02-17T12:06:03.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:03.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:06:03.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397507 2026-02-17T12:06:03.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:03.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397507 2026-02-17T12:06:03.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397507' 2026-02-17T12:06:03.082 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.10 seq 210453397507 2026-02-17T12:06:03.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:06:03.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397508 -lt 210453397507 2026-02-17T12:06:03.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:03.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:03.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:03.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:06:03.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:03.525 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:03.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:03.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:03.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:03.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:03.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:03.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:06:03.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:03.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:03.715 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:03.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:06:03.975 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:03.976 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:06:03.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.63880/ceph-osd.0.asok 2026-02-17T12:06:03.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-17T12:06:03.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.63880/ceph-osd.0.asok log flush 2026-02-17T12:06:04.020 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:06:04.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-17T12:06:04.026 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:04:23.745+0000 7f72cb8c6780 0 load: jerasure load: lrc 2026-02-17T12:06:04.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-17T12:06:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-17T12:06:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-17T12:06:04.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-17T12:06:04.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-17T12:06:04.656 INFO:tasks.workunit.client.0.trial044.stderr:pool 'ecpool' already exists 2026-02-17T12:06:04.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:06:05.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:06:05.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:05.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:05.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:05.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:05.669 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:05.669 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:05.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:05.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:05.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:05.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:05.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:05.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:05.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:05.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:05.751 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:05.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:05.941 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:06:05.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:05.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:05.942 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:06.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836504 2026-02-17T12:06:06.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836504 2026-02-17T12:06:06.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504' 2026-02-17T12:06:06.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.004 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:06.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672984 2026-02-17T12:06:06.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672984 2026-02-17T12:06:06.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984' 2026-02-17T12:06:06.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.064 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:06.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542165 2026-02-17T12:06:06.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542165 2026-02-17T12:06:06.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165' 2026-02-17T12:06:06.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:06:06.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411347 2026-02-17T12:06:06.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411347 2026-02-17T12:06:06.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347' 2026-02-17T12:06:06.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.188 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:06:06.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247826 2026-02-17T12:06:06.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247826 2026-02-17T12:06:06.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826' 2026-02-17T12:06:06.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.252 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:06:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117007 2026-02-17T12:06:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117007 2026-02-17T12:06:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007' 2026-02-17T12:06:06.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:06:06.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953486 2026-02-17T12:06:06.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953486 2026-02-17T12:06:06.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007 6-137438953486' 2026-02-17T12:06:06.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.374 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:06:06.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822669 2026-02-17T12:06:06.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822669 2026-02-17T12:06:06.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007 6-137438953486 7-154618822669' 2026-02-17T12:06:06.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.437 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:06:06.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659146 2026-02-17T12:06:06.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659146 2026-02-17T12:06:06.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007 6-137438953486 7-154618822669 8-176093659146' 2026-02-17T12:06:06.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:06:06.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528329 2026-02-17T12:06:06.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528329 2026-02-17T12:06:06.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007 6-137438953486 7-154618822669 8-176093659146 9-193273528329' 2026-02-17T12:06:06.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:06.559 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:06:06.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397510 2026-02-17T12:06:06.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397510 2026-02-17T12:06:06.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-42949672984 2-60129542165 3-77309411347 4-98784247826 5-115964117007 6-137438953486 7-154618822669 8-176093659146 9-193273528329 10-210453397510' 2026-02-17T12:06:06.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:06.622 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836504 2026-02-17T12:06:06.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:06.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:06.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836504 2026-02-17T12:06:06.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:06.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836504 2026-02-17T12:06:06.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-17T12:06:06.626 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836504 2026-02-17T12:06:06.626 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:06.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836502 -lt 21474836504 2026-02-17T12:06:06.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:07.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:07.814 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:08.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836504 2026-02-17T12:06:08.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.003 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672984 2026-02-17T12:06:08.003 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:08.004 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672984 2026-02-17T12:06:08.004 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672984 2026-02-17T12:06:08.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672984' 2026-02-17T12:06:08.006 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 42949672984 2026-02-17T12:06:08.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:08.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672984 -lt 42949672984 2026-02-17T12:06:08.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542165 2026-02-17T12:06:08.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:08.194 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542165 2026-02-17T12:06:08.194 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542165 2026-02-17T12:06:08.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542165' 2026-02-17T12:06:08.196 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 60129542165 2026-02-17T12:06:08.196 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:08.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542165 -lt 60129542165 2026-02-17T12:06:08.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411347 2026-02-17T12:06:08.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:08.387 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411347 2026-02-17T12:06:08.387 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411347 2026-02-17T12:06:08.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411347' 2026-02-17T12:06:08.388 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 77309411347 2026-02-17T12:06:08.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:08.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411347 -lt 77309411347 2026-02-17T12:06:08.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.577 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247826 2026-02-17T12:06:08.578 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:08.579 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247826 2026-02-17T12:06:08.580 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247826 2026-02-17T12:06:08.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247826' 2026-02-17T12:06:08.581 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 98784247826 2026-02-17T12:06:08.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:08.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247826 -lt 98784247826 2026-02-17T12:06:08.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.770 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117007 2026-02-17T12:06:08.770 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:06:08.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117007 2026-02-17T12:06:08.772 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117007 2026-02-17T12:06:08.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117007' 2026-02-17T12:06:08.773 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 115964117007 2026-02-17T12:06:08.773 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:06:08.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117008 -lt 115964117007 2026-02-17T12:06:08.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:08.961 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-137438953486 2026-02-17T12:06:08.961 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:08.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:06:08.962 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-137438953486 2026-02-17T12:06:08.962 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:08.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953486 2026-02-17T12:06:08.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 137438953486' 2026-02-17T12:06:08.964 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 137438953486 2026-02-17T12:06:08.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:06:09.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953486 -lt 137438953486 2026-02-17T12:06:09.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:09.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822669 2026-02-17T12:06:09.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:09.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:06:09.153 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822669 2026-02-17T12:06:09.153 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:09.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822669 2026-02-17T12:06:09.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822669' 2026-02-17T12:06:09.154 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 154618822669 2026-02-17T12:06:09.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:06:09.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822669 -lt 154618822669 2026-02-17T12:06:09.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:09.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659146 2026-02-17T12:06:09.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:09.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:06:09.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659146 2026-02-17T12:06:09.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:09.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659146 2026-02-17T12:06:09.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659146' 2026-02-17T12:06:09.347 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 176093659146 2026-02-17T12:06:09.348 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:06:09.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659147 -lt 176093659146 2026-02-17T12:06:09.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:09.536 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528329 2026-02-17T12:06:09.536 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:09.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:06:09.538 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528329 2026-02-17T12:06:09.538 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:09.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528329 2026-02-17T12:06:09.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528329' 2026-02-17T12:06:09.539 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.9 seq 193273528329 2026-02-17T12:06:09.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:06:09.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528330 -lt 193273528329 2026-02-17T12:06:09.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:09.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397510 2026-02-17T12:06:09.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:09.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:06:09.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397510 2026-02-17T12:06:09.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:09.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397510 2026-02-17T12:06:09.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397510' 2026-02-17T12:06:09.740 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.10 seq 210453397510 2026-02-17T12:06:09.740 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:06:09.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397511 -lt 210453397510 2026-02-17T12:06:09.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:09.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:09.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:10.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T12:06:10.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:10.176 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:10.176 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:10.177 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:10.177 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:10.177 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:10.177 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:10.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T12:06:10.367 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:10.368 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:10.368 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:10.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T12:06:10.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:10.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:10.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-17T12:06:10.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:06:10.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-17T12:06:10.630 INFO:tasks.workunit.client.0.trial044.stderr:TEST_chunk_mapping 2026-02-17T12:06:10.630 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_put_get_isa 2026-02-17T12:06:10.630 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_put_get_jerasure 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_put_get_lrc_advanced 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_put_get_lrc_kml 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_put_get_shec' 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: TEST_alignment_constraints: local payload=ABC 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: TEST_alignment_constraints: echo ABC 2026-02-17T12:06:10.631 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-17T12:06:10.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: local stripe_unit=4096 2026-02-17T12:06:10.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-17T12:06:10.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: grep k= 2026-02-17T12:06:10.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: eval local k=2 2026-02-17T12:06:10.839 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: local k=2 2026-02-17T12:06:10.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: TEST_alignment_constraints: local block_size=8191 2026-02-17T12:06:10.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:263: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-17T12:06:10.840 INFO:tasks.workunit.client.0.trial044.stderr:2+0 records in 2026-02-17T12:06:10.840 INFO:tasks.workunit.client.0.trial044.stderr:2+0 records out 2026-02-17T12:06:10.840 INFO:tasks.workunit.client.0.trial044.stderr:16382 bytes (16 kB, 16 KiB) copied, 7.0901e-05 s, 231 MB/s 2026-02-17T12:06:10.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:264: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-17T12:06:10.856 INFO:tasks.workunit.client.0.trial044.stderr:INFO: op_size has been rounded to 8192 2026-02-17T12:06:10.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:06:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:06:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-17T12:06:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-17T12:06:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:311: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-17T12:06:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-17T12:06:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=ecpool 2026-02-17T12:06:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=0 2026-02-17T12:06:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=1 2026-02-17T12:06:10.863 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-17T12:06:10.863 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-17T12:06:10.872 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-17T12:06:10.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-17T12:06:10.873 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-17T12:06:10.873 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-17T12:06:10.883 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-17T12:06:10.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-17T12:06:10.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-17T12:06:10.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-17T12:06:10.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-17T12:06:10.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-17T12:06:10.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-17T12:06:10.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:06:10.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHINGecpool 2026-02-17T12:06:10.923 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-17T12:06:10.923 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr:0' 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-17T12:06:11.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-17T12:06:11.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-17T12:06:11.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-17T12:06:11.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-17T12:06:11.174 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:11.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:11.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-17T12:06:11.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-17T12:06:11.232 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:11.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:11.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-17T12:06:11.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-17T12:06:11.292 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:11.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:11.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-17T12:06:11.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-17T12:06:11.348 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:11.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:11.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-17T12:06:11.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:06:11.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-17T12:06:11.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-17T12:06:11.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:06:11.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHINGecpool 2026-02-17T12:06:11.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-17T12:06:11.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:11.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:06:11.543 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:0' 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-code 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:06:11.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:11.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:12.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:12.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-17T12:06:12.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-17T12:06:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-17T12:06:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-code 2026-02-17T12:06:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:06:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:13.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:13.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:start osd.3 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:13.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-17T12:06:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:06:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:13.444 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:13.443+0000 7f8ccd282780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:13.445 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:13.444+0000 7f8ccd282780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:13.446 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:13.445+0000 7f8ccd282780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:13.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:06:13.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:13.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:13.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:06:13.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:14.789 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:14.788+0000 7f8ccd282780 -1 Falling back to public interface 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:14.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:06:14.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:15.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:06:16.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:17.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:17.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:17.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:17.189 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:17.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:17.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:06:17.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:17.491 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:17.489+0000 7f8ccd282780 -1 osd.3 64 log_to_monitors true 2026-02-17T12:06:18.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:18.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:18.382 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:06:18.582 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [18,64) [v2:127.0.0.1:6826/819946707,v1:127.0.0.1:6827/819946707] [v2:127.0.0.1:6828/819946707,v1:127.0.0.1:6829/819946707] exists,up 13436238-57c1-4815-b8b9-922dc46198c9 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:18.583 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:18.584 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:18.584 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:18.584 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:18.584 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:18.584 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:18.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:18.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:18.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:18.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:18.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:18.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:18.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:18.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:18.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836509 2026-02-17T12:06:18.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836509 2026-02-17T12:06:18.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509' 2026-02-17T12:06:18.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:18.916 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:18.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672988 2026-02-17T12:06:18.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672988 2026-02-17T12:06:18.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988' 2026-02-17T12:06:18.976 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:18.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:19.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542169 2026-02-17T12:06:19.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542169 2026-02-17T12:06:19.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169' 2026-02-17T12:06:19.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:06:19.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776131 2026-02-17T12:06:19.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776131 2026-02-17T12:06:19.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131' 2026-02-17T12:06:19.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.100 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:06:19.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247831 2026-02-17T12:06:19.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247831 2026-02-17T12:06:19.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831' 2026-02-17T12:06:19.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:06:19.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117012 2026-02-17T12:06:19.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117012 2026-02-17T12:06:19.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012' 2026-02-17T12:06:19.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:06:19.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953490 2026-02-17T12:06:19.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953490 2026-02-17T12:06:19.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012 6-137438953490' 2026-02-17T12:06:19.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:06:19.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822673 2026-02-17T12:06:19.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822673 2026-02-17T12:06:19.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012 6-137438953490 7-154618822673' 2026-02-17T12:06:19.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:06:19.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659151 2026-02-17T12:06:19.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659151 2026-02-17T12:06:19.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012 6-137438953490 7-154618822673 8-176093659151' 2026-02-17T12:06:19.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:06:19.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528334 2026-02-17T12:06:19.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528334 2026-02-17T12:06:19.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012 6-137438953490 7-154618822673 8-176093659151 9-193273528334' 2026-02-17T12:06:19.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:19.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397515 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397515 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-60129542169 3-292057776131 4-98784247831 5-115964117012 6-137438953490 7-154618822673 8-176093659151 9-193273528334 10-210453397515' 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836509 2026-02-17T12:06:19.532 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:19.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:19.533 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836509 2026-02-17T12:06:19.533 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:19.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836509 2026-02-17T12:06:19.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-17T12:06:19.535 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 21474836509 2026-02-17T12:06:19.535 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:19.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836509 -lt 21474836509 2026-02-17T12:06:19.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:19.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672988 2026-02-17T12:06:19.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:19.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:19.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672988 2026-02-17T12:06:19.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:19.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672988 2026-02-17T12:06:19.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672988' 2026-02-17T12:06:19.728 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 42949672988 2026-02-17T12:06:19.728 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:19.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672988 -lt 42949672988 2026-02-17T12:06:19.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:19.917 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542169 2026-02-17T12:06:19.917 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:19.919 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542169 2026-02-17T12:06:19.919 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:19.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542169 2026-02-17T12:06:19.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542169' 2026-02-17T12:06:19.920 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 60129542169 2026-02-17T12:06:19.920 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:20.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542169 -lt 60129542169 2026-02-17T12:06:20.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:20.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776131 2026-02-17T12:06:20.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:20.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:20.110 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776131 2026-02-17T12:06:20.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:20.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776131 2026-02-17T12:06:20.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776131' 2026-02-17T12:06:20.112 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 292057776131 2026-02-17T12:06:20.112 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:20.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 292057776131 2026-02-17T12:06:20.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:21.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:21.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:21.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-17T12:06:21.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:21.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247831 2026-02-17T12:06:21.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:21.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:21.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247831 2026-02-17T12:06:21.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:21.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247831 2026-02-17T12:06:21.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247831' 2026-02-17T12:06:21.496 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.4 seq 98784247831 2026-02-17T12:06:21.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:21.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247831 -lt 98784247831 2026-02-17T12:06:21.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:21.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117012 2026-02-17T12:06:21.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:21.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:06:21.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117012 2026-02-17T12:06:21.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:21.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117012 2026-02-17T12:06:21.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117012' 2026-02-17T12:06:21.687 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.5 seq 115964117012 2026-02-17T12:06:21.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:06:21.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117012 -lt 115964117012 2026-02-17T12:06:21.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:21.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-137438953490 2026-02-17T12:06:21.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:21.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:06:21.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-137438953490 2026-02-17T12:06:21.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:21.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953490 2026-02-17T12:06:21.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 137438953490' 2026-02-17T12:06:21.878 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.6 seq 137438953490 2026-02-17T12:06:21.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:06:22.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953491 -lt 137438953490 2026-02-17T12:06:22.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:22.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822673 2026-02-17T12:06:22.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:22.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:06:22.069 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822673 2026-02-17T12:06:22.069 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:22.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822673 2026-02-17T12:06:22.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822673' 2026-02-17T12:06:22.071 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.7 seq 154618822673 2026-02-17T12:06:22.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:06:22.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822674 -lt 154618822673 2026-02-17T12:06:22.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:22.256 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659151 2026-02-17T12:06:22.256 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:06:22.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659151 2026-02-17T12:06:22.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:22.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659151 2026-02-17T12:06:22.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659151' 2026-02-17T12:06:22.258 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.8 seq 176093659151 2026-02-17T12:06:22.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:06:22.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659151 -lt 176093659151 2026-02-17T12:06:22.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:22.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528334 2026-02-17T12:06:22.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:22.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:06:22.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528334 2026-02-17T12:06:22.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:22.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528334 2026-02-17T12:06:22.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528334' 2026-02-17T12:06:22.452 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.9 seq 193273528334 2026-02-17T12:06:22.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:06:22.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528334 -lt 193273528334 2026-02-17T12:06:22.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:22.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397515 2026-02-17T12:06:22.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:22.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:06:22.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397515 2026-02-17T12:06:22.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:22.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397515 2026-02-17T12:06:22.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397515' 2026-02-17T12:06:22.642 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.10 seq 210453397515 2026-02-17T12:06:22.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:06:22.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397515 -lt 210453397515 2026-02-17T12:06:22.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:22.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:22.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:23.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T12:06:23.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:23.084 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:23.084 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:23.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:23.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:23.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:23.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:23.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T12:06:23.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:23.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:23.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:23.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T12:06:23.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:23.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:23.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-17T12:06:23.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-code 2026-02-17T12:06:23.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:23.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:23.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:23.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:23.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:24.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-17T12:06:24.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-code 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-17T12:06:25.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:25.408 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:25.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr:start osd.5 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-17T12:06:25.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T12:06:25.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:25.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:25.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:25.422 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:25.421+0000 7f2e04e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:25.423 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:25.422+0000 7f2e04e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:25.424 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:25.423+0000 7f2e04e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:25.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:25.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:25.597 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:06:25.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:25.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:06:25.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:26.757 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:26.755+0000 7f2e04e0d780 -1 Falling back to public interface 2026-02-17T12:06:26.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:26.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:26.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:26.782 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:26.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:26.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:06:26.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:27.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:27.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:27.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:27.972 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:27.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:27.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:06:28.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:29.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:29.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:29.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:29.158 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:29.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:29.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:06:29.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:29.730 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:29.728+0000 7f2e04e0d780 -1 osd.5 70 log_to_monitors true 2026-02-17T12:06:30.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:30.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:30.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:30.348 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:30.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:30.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:06:30.537 INFO:tasks.workunit.client.0.trial044.stderr:osd.5 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [27,70) [v2:127.0.0.1:6842/2300785766,v1:127.0.0.1:6843/2300785766] [v2:127.0.0.1:6844/2300785766,v1:127.0.0.1:6845/2300785766] exists,up 4619773a-ed76-4068-a2ec-d89c6166d738 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:30.538 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:30.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:30.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:30.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:30.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:30.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:30.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:30.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:30.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:30.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:30.809 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:30.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:30.811 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:30.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T12:06:30.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T12:06:30.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T12:06:30.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:30.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:30.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672993 2026-02-17T12:06:30.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672993 2026-02-17T12:06:30.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993' 2026-02-17T12:06:30.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:30.932 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:30.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542174 2026-02-17T12:06:30.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542174 2026-02-17T12:06:30.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174' 2026-02-17T12:06:30.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:30.994 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:06:31.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776135 2026-02-17T12:06:31.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776135 2026-02-17T12:06:31.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135' 2026-02-17T12:06:31.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.056 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:06:31.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247835 2026-02-17T12:06:31.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247835 2026-02-17T12:06:31.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835' 2026-02-17T12:06:31.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.118 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:06:31.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579907 2026-02-17T12:06:31.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579907 2026-02-17T12:06:31.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907' 2026-02-17T12:06:31.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.179 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:06:31.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953495 2026-02-17T12:06:31.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953495 2026-02-17T12:06:31.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907 6-137438953495' 2026-02-17T12:06:31.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.241 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:06:31.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822678 2026-02-17T12:06:31.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822678 2026-02-17T12:06:31.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907 6-137438953495 7-154618822678' 2026-02-17T12:06:31.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:06:31.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659155 2026-02-17T12:06:31.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659155 2026-02-17T12:06:31.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907 6-137438953495 7-154618822678 8-176093659155' 2026-02-17T12:06:31.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.364 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:06:31.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528338 2026-02-17T12:06:31.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528338 2026-02-17T12:06:31.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907 6-137438953495 7-154618822678 8-176093659155 9-193273528338' 2026-02-17T12:06:31.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:31.427 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397519 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397519 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672993 2-60129542174 3-292057776135 4-98784247835 5-317827579907 6-137438953495 7-154618822678 8-176093659155 9-193273528338 10-210453397519' 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T12:06:31.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:31.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:31.490 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T12:06:31.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:31.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T12:06:31.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T12:06:31.492 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 21474836513 2026-02-17T12:06:31.492 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:31.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836513 2026-02-17T12:06:31.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:31.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672993 2026-02-17T12:06:31.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:31.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:31.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672993 2026-02-17T12:06:31.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:31.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672993 2026-02-17T12:06:31.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672993' 2026-02-17T12:06:31.690 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 42949672993 2026-02-17T12:06:31.690 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:31.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672993 -lt 42949672993 2026-02-17T12:06:31.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:31.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542174 2026-02-17T12:06:31.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:31.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:31.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542174 2026-02-17T12:06:31.886 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:31.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542174 2026-02-17T12:06:31.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542174' 2026-02-17T12:06:31.887 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 60129542174 2026-02-17T12:06:31.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:32.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542174 -lt 60129542174 2026-02-17T12:06:32.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:32.075 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776135 2026-02-17T12:06:32.075 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:32.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:32.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776135 2026-02-17T12:06:32.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:32.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776135 2026-02-17T12:06:32.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776135' 2026-02-17T12:06:32.078 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 292057776135 2026-02-17T12:06:32.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:32.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776135 -lt 292057776135 2026-02-17T12:06:32.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:32.267 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247835 2026-02-17T12:06:32.267 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:32.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:32.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247835 2026-02-17T12:06:32.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:32.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247835 2026-02-17T12:06:32.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247835' 2026-02-17T12:06:32.269 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.4 seq 98784247835 2026-02-17T12:06:32.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:32.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247833 -lt 98784247835 2026-02-17T12:06:32.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:33.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:33.468 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:33.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247836 -lt 98784247835 2026-02-17T12:06:33.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:33.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579907 2026-02-17T12:06:33.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:33.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:06:33.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579907 2026-02-17T12:06:33.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:33.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579907 2026-02-17T12:06:33.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579907' 2026-02-17T12:06:33.659 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.5 seq 317827579907 2026-02-17T12:06:33.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:06:33.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-17T12:06:33.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:33.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-137438953495 2026-02-17T12:06:33.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:33.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:06:33.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-137438953495 2026-02-17T12:06:33.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:33.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953495 2026-02-17T12:06:33.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 137438953495' 2026-02-17T12:06:33.858 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.6 seq 137438953495 2026-02-17T12:06:33.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:06:34.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953495 -lt 137438953495 2026-02-17T12:06:34.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:34.047 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822678 2026-02-17T12:06:34.047 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:34.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:06:34.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822678 2026-02-17T12:06:34.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:34.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822678 2026-02-17T12:06:34.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822678' 2026-02-17T12:06:34.049 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.7 seq 154618822678 2026-02-17T12:06:34.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:06:34.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822678 -lt 154618822678 2026-02-17T12:06:34.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:34.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659155 2026-02-17T12:06:34.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:34.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:06:34.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659155 2026-02-17T12:06:34.241 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:34.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659155 2026-02-17T12:06:34.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659155' 2026-02-17T12:06:34.242 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.8 seq 176093659155 2026-02-17T12:06:34.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:06:34.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659155 -lt 176093659155 2026-02-17T12:06:34.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:34.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528338 2026-02-17T12:06:34.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:34.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:06:34.436 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528338 2026-02-17T12:06:34.436 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:34.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528338 2026-02-17T12:06:34.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528338' 2026-02-17T12:06:34.437 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.9 seq 193273528338 2026-02-17T12:06:34.437 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:06:34.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528338 -lt 193273528338 2026-02-17T12:06:34.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:34.626 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397519 2026-02-17T12:06:34.626 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:34.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:06:34.627 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397519 2026-02-17T12:06:34.627 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:34.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397519 2026-02-17T12:06:34.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397519' 2026-02-17T12:06:34.628 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.10 seq 210453397519 2026-02-17T12:06:34.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:06:34.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397520 -lt 210453397519 2026-02-17T12:06:34.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:34.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:34.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:35.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T12:06:35.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:35.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:35.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:35.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:35.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:35.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:35.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:35.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T12:06:35.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:35.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:35.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:35.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T12:06:35.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:35.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:35.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-17T12:06:35.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:318: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-17T12:06:35.934 INFO:tasks.workunit.client.0.trial044.stdout:crush-device-class= 2026-02-17T12:06:35.934 INFO:tasks.workunit.client.0.trial044.stdout:crush-failure-domain=host 2026-02-17T12:06:35.934 INFO:tasks.workunit.client.0.trial044.stdout:crush-num-failure-domains=0 2026-02-17T12:06:35.934 INFO:tasks.workunit.client.0.trial044.stdout:crush-osds-per-failure-domain=0 2026-02-17T12:06:35.934 INFO:tasks.workunit.client.0.trial044.stdout:crush-root=default 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:k=-1 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:l=-1 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:layers=[ [ "cDD", "" ] ] 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:m=-1 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:mapping=_DD 2026-02-17T12:06:35.935 INFO:tasks.workunit.client.0.trial044.stdout:plugin=lrc 2026-02-17T12:06:35.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:319: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-17T12:06:35.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-17T12:06:36.238 INFO:tasks.workunit.client.0.trial044.stderr:pool 'remap-pool' already exists 2026-02-17T12:06:36.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:06:37.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-17T12:06:37.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-17T12:06:37.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=remap-pool 2026-02-17T12:06:37.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=1 2026-02-17T12:06:37.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=2 2026-02-17T12:06:37.249 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-17T12:06:37.250 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-17T12:06:37.261 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-17T12:06:37.261 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-17T12:06:37.261 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-17T12:06:37.262 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-17T12:06:37.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-17T12:06:37.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-17T12:06:37.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-17T12:06:37.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-17T12:06:37.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-17T12:06:37.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-17T12:06:37.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-17T12:06:37.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=remap-pool 2026-02-17T12:06:37.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHINGremap-pool 2026-02-17T12:06:37.314 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-17T12:06:37.314 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=4 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr:9' 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 4 6 9 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-17T12:06:37.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-17T12:06:37.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-17T12:06:37.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-17T12:06:37.572 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:37.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:37.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-17T12:06:37.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-17T12:06:37.630 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:37.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:37.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-17T12:06:37.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-17T12:06:37.689 INFO:tasks.workunit.client.0.trial044.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-17T12:06:37.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-17T12:06:37.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-17T12:06:37.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:06:37.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-17T12:06:37.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-17T12:06:37.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=remap-pool 2026-02-17T12:06:37.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHINGremap-pool 2026-02-17T12:06:37.693 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-17T12:06:37.693 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=4 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr:9' 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 4 6 9 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-17T12:06:37.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-code 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=6 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=6 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-17T12:06:37.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:37.891 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:37.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:37.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:37.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=6 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-17T12:06:39.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:39.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-code 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=6 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:39.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:39.765 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:39.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-17T12:06:39.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.6 2026-02-17T12:06:39.767 INFO:tasks.workunit.client.0.trial044.stderr:start osd.6 2026-02-17T12:06:39.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 6 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:39.767 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-17T12:06:39.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 6 = 6 ']' 2026-02-17T12:06:39.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:39.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:39.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:39.779 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:39.778+0000 7fc3a9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:39.780 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:39.779+0000 7fc3a9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:39.781 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:39.780+0000 7fc3a9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 6 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:39.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:39.954 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:06:39.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:39.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:06:40.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:41.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:41.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:41.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:41.145 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:41.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:41.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:06:41.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:41.364 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:41.362+0000 7fc3a9816780 -1 Falling back to public interface 2026-02-17T12:06:42.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:42.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:42.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:42.335 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:42.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:42.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:06:42.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:43.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:06:43.715 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:44.086 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:44.084+0000 7fc3a9816780 -1 osd.6 81 log_to_monitors true 2026-02-17T12:06:44.715 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:44.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:44.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:44.716 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:44.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:44.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:06:44.913 INFO:tasks.workunit.client.0.trial044.stderr:osd.6 up in weight 1 up_from 85 up_thru 85 down_at 82 last_clean_interval [32,81) [v2:127.0.0.1:6850/544268311,v1:127.0.0.1:6851/544268311] [v2:127.0.0.1:6852/544268311,v1:127.0.0.1:6853/544268311] exists,up 1ecc33f4-0f1a-4601-9a7b-1661b69af4c6 2026-02-17T12:06:44.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:44.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:44.915 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:44.915 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:44.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:44.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:44.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:44.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:44.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:44.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:44.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:44.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:44.996 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:45.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:45.182 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.183 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:45.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836518 2026-02-17T12:06:45.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836518 2026-02-17T12:06:45.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518' 2026-02-17T12:06:45.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:45.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672997 2026-02-17T12:06:45.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672997 2026-02-17T12:06:45.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997' 2026-02-17T12:06:45.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:45.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542179 2026-02-17T12:06:45.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542179 2026-02-17T12:06:45.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179' 2026-02-17T12:06:45.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.372 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:06:45.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776140 2026-02-17T12:06:45.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776140 2026-02-17T12:06:45.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140' 2026-02-17T12:06:45.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:06:45.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247840 2026-02-17T12:06:45.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247840 2026-02-17T12:06:45.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840' 2026-02-17T12:06:45.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:06:45.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579912 2026-02-17T12:06:45.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579912 2026-02-17T12:06:45.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912' 2026-02-17T12:06:45.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.556 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:06:45.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220163 2026-02-17T12:06:45.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220163 2026-02-17T12:06:45.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912 6-365072220163' 2026-02-17T12:06:45.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:06:45.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822683 2026-02-17T12:06:45.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822683 2026-02-17T12:06:45.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912 6-365072220163 7-154618822683' 2026-02-17T12:06:45.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.680 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:06:45.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659160 2026-02-17T12:06:45.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659160 2026-02-17T12:06:45.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912 6-365072220163 7-154618822683 8-176093659160' 2026-02-17T12:06:45.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:06:45.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528343 2026-02-17T12:06:45.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528343 2026-02-17T12:06:45.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912 6-365072220163 7-154618822683 8-176093659160 9-193273528343' 2026-02-17T12:06:45.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:45.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:06:45.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397524 2026-02-17T12:06:45.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397524 2026-02-17T12:06:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-60129542179 3-292057776140 4-98784247840 5-317827579912 6-365072220163 7-154618822683 8-176093659160 9-193273528343 10-210453397524' 2026-02-17T12:06:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:45.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836518 2026-02-17T12:06:45.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:45.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836518 2026-02-17T12:06:45.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:45.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836518 2026-02-17T12:06:45.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-17T12:06:45.869 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 21474836518 2026-02-17T12:06:45.869 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:46.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-17T12:06:46.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:47.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:47.061 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:47.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836518 -lt 21474836518 2026-02-17T12:06:47.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:47.261 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672997 2026-02-17T12:06:47.261 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:47.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:47.263 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672997 2026-02-17T12:06:47.263 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:47.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672997 2026-02-17T12:06:47.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672997' 2026-02-17T12:06:47.264 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 42949672997 2026-02-17T12:06:47.264 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:47.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672998 -lt 42949672997 2026-02-17T12:06:47.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:47.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542179 2026-02-17T12:06:47.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:47.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:47.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542179 2026-02-17T12:06:47.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:47.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542179 2026-02-17T12:06:47.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542179' 2026-02-17T12:06:47.462 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 60129542179 2026-02-17T12:06:47.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:47.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542179 -lt 60129542179 2026-02-17T12:06:47.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:47.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776140 2026-02-17T12:06:47.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:47.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:47.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776140 2026-02-17T12:06:47.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:47.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776140 2026-02-17T12:06:47.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776140' 2026-02-17T12:06:47.652 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 292057776140 2026-02-17T12:06:47.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:47.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776140 -lt 292057776140 2026-02-17T12:06:47.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:47.839 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247840 2026-02-17T12:06:47.839 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:47.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:47.841 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247840 2026-02-17T12:06:47.841 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:47.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247840 2026-02-17T12:06:47.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247840' 2026-02-17T12:06:47.842 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.4 seq 98784247840 2026-02-17T12:06:47.842 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:06:48.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247840 -lt 98784247840 2026-02-17T12:06:48.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:48.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579912 2026-02-17T12:06:48.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:48.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:06:48.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579912 2026-02-17T12:06:48.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:48.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579912 2026-02-17T12:06:48.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579912' 2026-02-17T12:06:48.035 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.5 seq 317827579912 2026-02-17T12:06:48.035 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:06:48.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579912 -lt 317827579912 2026-02-17T12:06:48.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:48.227 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-365072220163 2026-02-17T12:06:48.227 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:48.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:06:48.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-365072220163 2026-02-17T12:06:48.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:48.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220163 2026-02-17T12:06:48.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 365072220163' 2026-02-17T12:06:48.230 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.6 seq 365072220163 2026-02-17T12:06:48.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:06:48.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220163 -lt 365072220163 2026-02-17T12:06:48.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:48.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822683 2026-02-17T12:06:48.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:48.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:06:48.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822683 2026-02-17T12:06:48.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:48.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822683 2026-02-17T12:06:48.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822683' 2026-02-17T12:06:48.425 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.7 seq 154618822683 2026-02-17T12:06:48.425 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:06:48.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822683 -lt 154618822683 2026-02-17T12:06:48.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:48.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659160 2026-02-17T12:06:48.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:48.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:06:48.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659160 2026-02-17T12:06:48.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:48.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659160 2026-02-17T12:06:48.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659160' 2026-02-17T12:06:48.625 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.8 seq 176093659160 2026-02-17T12:06:48.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:06:48.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659160 -lt 176093659160 2026-02-17T12:06:48.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:48.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528343 2026-02-17T12:06:48.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:48.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:06:48.818 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528343 2026-02-17T12:06:48.818 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:48.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528343 2026-02-17T12:06:48.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528343' 2026-02-17T12:06:48.819 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.9 seq 193273528343 2026-02-17T12:06:48.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:06:49.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528343 -lt 193273528343 2026-02-17T12:06:49.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:49.008 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397524 2026-02-17T12:06:49.008 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:49.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:06:49.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397524 2026-02-17T12:06:49.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:49.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397524 2026-02-17T12:06:49.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397524' 2026-02-17T12:06:49.011 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.10 seq 210453397524 2026-02-17T12:06:49.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:06:49.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397525 -lt 210453397524 2026-02-17T12:06:49.201 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:49.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:49.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:49.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 == 0 2026-02-17T12:06:49.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:49.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:49.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:49.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:49.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:49.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:49.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:49.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=28 2026-02-17T12:06:49.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:49.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:49.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:49.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 28 = 28 2026-02-17T12:06:49.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:49.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-code 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T12:06:49.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:49.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:51.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-17T12:06:51.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-17T12:06:51.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-17T12:06:51.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-code 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-17T12:06:51.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:51.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:start osd.9 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=30876f99-cccb-48c3-8ce5-8383d11cea08 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.63880/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:51.782 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-17T12:06:51.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T12:06:51.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:51.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:51.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:51.793 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:51.792+0000 7f9c9b109780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:51.794 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:51.793+0000 7f9c9b109780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:51.795 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:51.794+0000 7f9c9b109780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:51.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:51.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:06:52.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:53.145 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:53.143+0000 7f9c9b109780 -1 Falling back to public interface 2026-02-17T12:06:53.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:53.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:53.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:53.165 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:53.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:53.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:06:53.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:54.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:54.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:54.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:54.355 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:54.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:54.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:06:54.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:55.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:55.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:55.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:55.544 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:55.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:55.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:06:55.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:55.879 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:06:55.878+0000 7f9c9b109780 -1 osd.9 87 log_to_monitors true 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:56.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T12:06:56.927 INFO:tasks.workunit.client.0.trial044.stderr:osd.9 up in weight 1 up_from 91 up_thru 91 down_at 88 last_clean_interval [45,87) [v2:127.0.0.1:6874/2019004232,v1:127.0.0.1:6875/2019004232] [v2:127.0.0.1:6876/2019004232,v1:127.0.0.1:6877/2019004232] exists,up c7f42f79-0655-43ec-b04a-ee7db982e022 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:56.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:57.007 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:06:57.193 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:06:57.194 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:06:57.194 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:06:57.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:57.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.194 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:57.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836522 2026-02-17T12:06:57.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836522 2026-02-17T12:06:57.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522' 2026-02-17T12:06:57.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:57.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673002 2026-02-17T12:06:57.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673002 2026-02-17T12:06:57.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002' 2026-02-17T12:06:57.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:57.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542183 2026-02-17T12:06:57.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542183 2026-02-17T12:06:57.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183' 2026-02-17T12:06:57.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.387 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:06:57.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776144 2026-02-17T12:06:57.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776144 2026-02-17T12:06:57.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144' 2026-02-17T12:06:57.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.446 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:06:57.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247844 2026-02-17T12:06:57.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247844 2026-02-17T12:06:57.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844' 2026-02-17T12:06:57.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:06:57.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579916 2026-02-17T12:06:57.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579916 2026-02-17T12:06:57.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916' 2026-02-17T12:06:57.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:06:57.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220167 2026-02-17T12:06:57.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220167 2026-02-17T12:06:57.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916 6-365072220167' 2026-02-17T12:06:57.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:06:57.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822687 2026-02-17T12:06:57.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822687 2026-02-17T12:06:57.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916 6-365072220167 7-154618822687' 2026-02-17T12:06:57.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:06:57.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659164 2026-02-17T12:06:57.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659164 2026-02-17T12:06:57.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916 6-365072220167 7-154618822687 8-176093659164' 2026-02-17T12:06:57.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:06:57.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023939 2026-02-17T12:06:57.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023939 2026-02-17T12:06:57.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916 6-365072220167 7-154618822687 8-176093659164 9-390842023939' 2026-02-17T12:06:57.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:57.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:06:57.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397529 2026-02-17T12:06:57.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397529 2026-02-17T12:06:57.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949673002 2-60129542183 3-292057776144 4-98784247844 5-317827579916 6-365072220167 7-154618822687 8-176093659164 9-390842023939 10-210453397529' 2026-02-17T12:06:57.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:57.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836522 2026-02-17T12:06:57.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:57.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:57.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836522 2026-02-17T12:06:57.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:57.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836522 2026-02-17T12:06:57.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-17T12:06:57.882 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 21474836522 2026-02-17T12:06:57.882 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:58.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836522 2026-02-17T12:06:58.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:59.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:59.072 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:59.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836523 -lt 21474836522 2026-02-17T12:06:59.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:59.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673002 2026-02-17T12:06:59.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:59.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:59.261 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673002 2026-02-17T12:06:59.261 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:59.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673002 2026-02-17T12:06:59.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673002' 2026-02-17T12:06:59.262 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 42949673002 2026-02-17T12:06:59.262 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:59.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673002 -lt 42949673002 2026-02-17T12:06:59.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:59.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542183 2026-02-17T12:06:59.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:59.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:59.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542183 2026-02-17T12:06:59.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:59.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542183 2026-02-17T12:06:59.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542183' 2026-02-17T12:06:59.455 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 60129542183 2026-02-17T12:06:59.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:59.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542183 -lt 60129542183 2026-02-17T12:06:59.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:59.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776144 2026-02-17T12:06:59.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:59.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:06:59.644 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776144 2026-02-17T12:06:59.644 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:59.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776144 2026-02-17T12:06:59.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776144' 2026-02-17T12:06:59.645 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 292057776144 2026-02-17T12:06:59.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:06:59.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776145 -lt 292057776144 2026-02-17T12:06:59.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:59.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247844 2026-02-17T12:06:59.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:59.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:06:59.837 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247844 2026-02-17T12:06:59.837 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:59.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247844 2026-02-17T12:06:59.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247844' 2026-02-17T12:06:59.838 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.4 seq 98784247844 2026-02-17T12:06:59.838 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:07:00.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247845 -lt 98784247844 2026-02-17T12:07:00.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579916 2026-02-17T12:07:00.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:07:00.029 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579916 2026-02-17T12:07:00.029 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579916 2026-02-17T12:07:00.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579916' 2026-02-17T12:07:00.030 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.5 seq 317827579916 2026-02-17T12:07:00.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:07:00.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579916 -lt 317827579916 2026-02-17T12:07:00.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-365072220167 2026-02-17T12:07:00.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:07:00.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-365072220167 2026-02-17T12:07:00.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220167 2026-02-17T12:07:00.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 365072220167' 2026-02-17T12:07:00.221 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.6 seq 365072220167 2026-02-17T12:07:00.221 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:07:00.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220167 -lt 365072220167 2026-02-17T12:07:00.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822687 2026-02-17T12:07:00.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:07:00.412 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822687 2026-02-17T12:07:00.412 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822687 2026-02-17T12:07:00.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822687' 2026-02-17T12:07:00.413 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.7 seq 154618822687 2026-02-17T12:07:00.413 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:07:00.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822687 -lt 154618822687 2026-02-17T12:07:00.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.601 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659164 2026-02-17T12:07:00.601 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:07:00.603 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659164 2026-02-17T12:07:00.603 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659164 2026-02-17T12:07:00.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659164' 2026-02-17T12:07:00.604 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.8 seq 176093659164 2026-02-17T12:07:00.604 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:07:00.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659165 -lt 176093659164 2026-02-17T12:07:00.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.794 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023939 2026-02-17T12:07:00.794 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:07:00.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023939 2026-02-17T12:07:00.796 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023939 2026-02-17T12:07:00.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023939' 2026-02-17T12:07:00.797 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.9 seq 390842023939 2026-02-17T12:07:00.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:07:00.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-17T12:07:00.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:00.988 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397529 2026-02-17T12:07:00.989 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:00.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:07:00.990 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397529 2026-02-17T12:07:00.990 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:00.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397529 2026-02-17T12:07:00.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397529' 2026-02-17T12:07:00.991 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.10 seq 210453397529 2026-02-17T12:07:00.991 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:07:01.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397529 -lt 210453397529 2026-02-17T12:07:01.179 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:01.180 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:01.180 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:01.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 == 0 2026-02-17T12:07:01.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:01.434 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:01.434 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:01.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:01.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:01.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:01.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:01.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=28 2026-02-17T12:07:01.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:01.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:01.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:01.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 28 = 28 2026-02-17T12:07:01.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:01.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:01.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:329: TEST_chunk_mapping: delete_pool remap-pool 2026-02-17T12:07:01.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=remap-pool 2026-02-17T12:07:01.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-17T12:07:02.083 INFO:tasks.workunit.client.0.trial044.stderr:pool 'remap-pool' does not exist 2026-02-17T12:07:02.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:330: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-17T12:07:02.288 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile remap-profile does not exist 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local plugin=isa 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: local status 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: local grepstr 2026-02-17T12:07:02.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: local s 2026-02-17T12:07:02.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2118: erasure_code_plugin_exists: case `uname` in 2026-02-17T12:07:02.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2118: erasure_code_plugin_exists: uname 2026-02-17T12:07:02.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2120: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-17T12:07:02.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2123: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-17T12:07:02.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2123: erasure_code_plugin_exists: s= 2026-02-17T12:07:02.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2124: erasure_code_plugin_exists: local status=0 2026-02-17T12:07:02.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-17T12:07:02.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2126: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-17T12:07:02.743 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-17T12:07:02.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: erasure_code_plugin_exists: return 0 2026-02-17T12:07:02.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-17T12:07:02.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-17T12:07:02.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-17T12:07:02.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-17T12:07:02.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-17T12:07:03.247 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-isa' already exists 2026-02-17T12:07:03.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:04.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-17T12:07:04.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:04.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:04.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:04.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:04.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:04.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:04.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-17T12:07:04.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-isa 2026-02-17T12:07:04.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:04.299 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-17T12:07:04.299 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=4 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr:10 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:04.488 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:9' 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 4 10 0 7 6 8 2 1 5 9 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7' '6' '8' '2' '1' '5' '9') 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=9 2026-02-17T12:07:04.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-17T12:07:04.729 INFO:tasks.workunit.client.0.trial044.stderr:osd.9 is already out. 2026-02-17T12:07:04.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:07:09.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-17T12:07:09.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-isa 2026-02-17T12:07:09.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:09.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-17T12:07:09.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-17T12:07:09.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:09.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=4 2026-02-17T12:07:09.933 INFO:tasks.workunit.client.0.trial044.stderr:10 2026-02-17T12:07:09.933 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:09.933 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:09.933 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 4 10 0 7 6 8 3 1 5 2 2026-02-17T12:07:09.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:09.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:09.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-17T12:07:10.197 INFO:tasks.workunit.client.0.trial044.stderr:osd.9 is already in. 2026-02-17T12:07:10.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:10.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-17T12:07:10.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=pool-isa 2026-02-17T12:07:10.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-17T12:07:10.499 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-isa' does not exist 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:10.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:10.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:10.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:10.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-17T12:07:10.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:07:10.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:10.550 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:07:10.550 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:10.740 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=6 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:9' 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 6 0 7 9 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-17T12:07:10.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-17T12:07:10.981 INFO:tasks.workunit.client.0.trial044.stderr:osd.9 is already out. 2026-02-17T12:07:10.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:07:15.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-17T12:07:15.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:07:15.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:15.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-17T12:07:15.991 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:07:15.992 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:16.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=6 2026-02-17T12:07:16.188 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:16.189 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:16.189 INFO:tasks.workunit.client.0.trial044.stderr:5' 2026-02-17T12:07:16.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 6 0 7 5 2026-02-17T12:07:16.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:16.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:16.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-17T12:07:16.452 INFO:tasks.workunit.client.0.trial044.stderr:osd.9 is already in. 2026-02-17T12:07:16.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:16.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-17T12:07:16.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-17T12:07:16.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-17T12:07:16.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-17T12:07:16.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-17T12:07:17.110 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:07:17.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:18.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-17T12:07:18.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:18.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-17T12:07:18.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:18.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:18.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:18.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:18.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:18.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:18.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:18.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:18.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:18.165 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:18.165 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:18.358 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 1 9 4 3 10 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-17T12:07:18.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-17T12:07:18.603 INFO:tasks.workunit.client.0.trial044.stderr:osd.10 is already out. 2026-02-17T12:07:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:07:23.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:23.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:23.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:23.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-17T12:07:23.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:23.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:23.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 1 9 4 3 6 2026-02-17T12:07:23.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:23.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:23.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-17T12:07:24.070 INFO:tasks.workunit.client.0.trial044.stderr:osd.10 is already in. 2026-02-17T12:07:24.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:24.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-17T12:07:24.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-17T12:07:24.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:24.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:24.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:24.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-17T12:07:24.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-17T12:07:24.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:24.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:24.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:24.326 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:24.327 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:24.327 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:24.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:24.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:24.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:24.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:24.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:24.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:24.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:24.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:24.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:24.598 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:07:24.599 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:24.599 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:24.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:24.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:24.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836530 2026-02-17T12:07:24.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836530 2026-02-17T12:07:24.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530' 2026-02-17T12:07:24.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.661 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:24.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673009 2026-02-17T12:07:24.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673009 2026-02-17T12:07:24.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009' 2026-02-17T12:07:24.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:07:24.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542190 2026-02-17T12:07:24.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542190 2026-02-17T12:07:24.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190' 2026-02-17T12:07:24.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.785 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776152 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776152 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152' 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:07:24.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247852 2026-02-17T12:07:24.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247852 2026-02-17T12:07:24.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852' 2026-02-17T12:07:24.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:07:24.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579924 2026-02-17T12:07:24.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579924 2026-02-17T12:07:24.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924' 2026-02-17T12:07:24.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:24.969 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:07:25.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220175 2026-02-17T12:07:25.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220175 2026-02-17T12:07:25.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924 6-365072220175' 2026-02-17T12:07:25.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:25.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:07:25.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822695 2026-02-17T12:07:25.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822695 2026-02-17T12:07:25.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924 6-365072220175 7-154618822695' 2026-02-17T12:07:25.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:25.095 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:07:25.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659172 2026-02-17T12:07:25.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659172 2026-02-17T12:07:25.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924 6-365072220175 7-154618822695 8-176093659172' 2026-02-17T12:07:25.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:25.156 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:07:25.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023946 2026-02-17T12:07:25.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023946 2026-02-17T12:07:25.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924 6-365072220175 7-154618822695 8-176093659172 9-390842023946' 2026-02-17T12:07:25.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:25.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:07:25.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397536 2026-02-17T12:07:25.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397536 2026-02-17T12:07:25.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673009 2-60129542190 3-292057776152 4-98784247852 5-317827579924 6-365072220175 7-154618822695 8-176093659172 9-390842023946 10-210453397536' 2026-02-17T12:07:25.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:25.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836530 2026-02-17T12:07:25.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:25.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:25.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836530 2026-02-17T12:07:25.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:25.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836530 2026-02-17T12:07:25.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836530' 2026-02-17T12:07:25.284 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836530 2026-02-17T12:07:25.284 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:25.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836530 2026-02-17T12:07:25.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:25.473 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673009 2026-02-17T12:07:25.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:25.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:25.475 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673009 2026-02-17T12:07:25.475 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:25.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673009 2026-02-17T12:07:25.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673009' 2026-02-17T12:07:25.476 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 42949673009 2026-02-17T12:07:25.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:25.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673009 -lt 42949673009 2026-02-17T12:07:25.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:25.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542190 2026-02-17T12:07:25.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:25.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:07:25.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542190 2026-02-17T12:07:25.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:25.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542190 2026-02-17T12:07:25.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542190' 2026-02-17T12:07:25.667 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 60129542190 2026-02-17T12:07:25.667 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:07:25.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542191 -lt 60129542190 2026-02-17T12:07:25.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:25.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776152 2026-02-17T12:07:25.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:25.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:07:25.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776152 2026-02-17T12:07:25.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:25.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776152 2026-02-17T12:07:25.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776152' 2026-02-17T12:07:25.860 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 292057776152 2026-02-17T12:07:25.860 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:07:26.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776152 -lt 292057776152 2026-02-17T12:07:26.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:26.051 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247852 2026-02-17T12:07:26.051 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:26.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:07:26.052 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247852 2026-02-17T12:07:26.052 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:26.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247852 2026-02-17T12:07:26.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247852' 2026-02-17T12:07:26.054 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 98784247852 2026-02-17T12:07:26.054 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:07:26.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247852 -lt 98784247852 2026-02-17T12:07:26.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:26.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579924 2026-02-17T12:07:26.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:26.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:07:26.247 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579924 2026-02-17T12:07:26.247 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:26.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579924 2026-02-17T12:07:26.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579924' 2026-02-17T12:07:26.249 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 317827579924 2026-02-17T12:07:26.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:07:26.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579924 -lt 317827579924 2026-02-17T12:07:26.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:26.440 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-365072220175 2026-02-17T12:07:26.440 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:26.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:07:26.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-365072220175 2026-02-17T12:07:26.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:26.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220175 2026-02-17T12:07:26.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 365072220175' 2026-02-17T12:07:26.443 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 365072220175 2026-02-17T12:07:26.444 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:07:26.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220175 -lt 365072220175 2026-02-17T12:07:26.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:26.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822695 2026-02-17T12:07:26.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:26.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:07:26.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822695 2026-02-17T12:07:26.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:26.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822695 2026-02-17T12:07:26.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822695' 2026-02-17T12:07:26.633 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 154618822695 2026-02-17T12:07:26.634 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:07:26.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822693 -lt 154618822695 2026-02-17T12:07:26.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:27.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:27.825 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:07:28.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822695 -lt 154618822695 2026-02-17T12:07:28.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:28.016 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659172 2026-02-17T12:07:28.016 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:28.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:07:28.018 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659172 2026-02-17T12:07:28.018 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:28.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659172 2026-02-17T12:07:28.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659172' 2026-02-17T12:07:28.019 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 176093659172 2026-02-17T12:07:28.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:07:28.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659172 -lt 176093659172 2026-02-17T12:07:28.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:28.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023946 2026-02-17T12:07:28.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:28.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:07:28.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023946 2026-02-17T12:07:28.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023946 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023946' 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.9 seq 390842023946 2026-02-17T12:07:28.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:07:28.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023947 -lt 390842023946 2026-02-17T12:07:28.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:28.403 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397536 2026-02-17T12:07:28.403 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:28.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:07:28.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397536 2026-02-17T12:07:28.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:28.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397536 2026-02-17T12:07:28.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397536' 2026-02-17T12:07:28.406 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.10 seq 210453397536 2026-02-17T12:07:28.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:07:28.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397537 -lt 210453397536 2026-02-17T12:07:28.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:28.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:28.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:28.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 == 0 2026-02-17T12:07:28.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:28.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:28.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:28.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:28.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:28.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:28.848 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:29.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=28 2026-02-17T12:07:29.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:29.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:29.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:29.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 28 = 28 2026-02-17T12:07:29.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:29.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:29.294 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:29.294 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:29.294 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:29.294 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:29.294 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 1 9 4 3 10 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-17T12:07:29.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-17T12:07:29.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-17T12:07:29.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:29.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-17T12:07:29.724 INFO:tasks.workunit.client.0.trial044.stderr:osd.5 is already out. 2026-02-17T12:07:29.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:29.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-17T12:07:29.935 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 is already out. 2026-02-17T12:07:29.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-17T12:07:29.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:29.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:29.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:29.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:29.945 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:29.945 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:29.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:29.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:29.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:30.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:30.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:30.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:30.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:30.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:30.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:30.220 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:30.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:30.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.221 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:30.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836533 2026-02-17T12:07:30.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836533 2026-02-17T12:07:30.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533' 2026-02-17T12:07:30.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.283 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:30.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673012 2026-02-17T12:07:30.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673012 2026-02-17T12:07:30.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012' 2026-02-17T12:07:30.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:07:30.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542194 2026-02-17T12:07:30.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542194 2026-02-17T12:07:30.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194' 2026-02-17T12:07:30.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:07:30.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776155 2026-02-17T12:07:30.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776155 2026-02-17T12:07:30.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155' 2026-02-17T12:07:30.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:07:30.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247855 2026-02-17T12:07:30.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247855 2026-02-17T12:07:30.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855' 2026-02-17T12:07:30.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.536 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:07:30.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579927 2026-02-17T12:07:30.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579927 2026-02-17T12:07:30.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927' 2026-02-17T12:07:30.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:07:30.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220178 2026-02-17T12:07:30.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220178 2026-02-17T12:07:30.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927 6-365072220178' 2026-02-17T12:07:30.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:07:30.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822698 2026-02-17T12:07:30.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822698 2026-02-17T12:07:30.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927 6-365072220178 7-154618822698' 2026-02-17T12:07:30.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.721 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:07:30.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659175 2026-02-17T12:07:30.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659175 2026-02-17T12:07:30.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927 6-365072220178 7-154618822698 8-176093659175' 2026-02-17T12:07:30.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.785 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:07:30.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023949 2026-02-17T12:07:30.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023949 2026-02-17T12:07:30.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927 6-365072220178 7-154618822698 8-176093659175 9-390842023949' 2026-02-17T12:07:30.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:30.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:07:30.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397539 2026-02-17T12:07:30.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397539 2026-02-17T12:07:30.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673012 2-60129542194 3-292057776155 4-98784247855 5-317827579927 6-365072220178 7-154618822698 8-176093659175 9-390842023949 10-210453397539' 2026-02-17T12:07:30.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:30.911 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836533 2026-02-17T12:07:30.912 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:30.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:30.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836533 2026-02-17T12:07:30.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:30.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836533 2026-02-17T12:07:30.914 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836533 2026-02-17T12:07:30.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-17T12:07:30.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:31.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-17T12:07:31.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:32.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:32.102 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:32.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836533 -lt 21474836533 2026-02-17T12:07:32.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:32.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673012 2026-02-17T12:07:32.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:32.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:32.292 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673012 2026-02-17T12:07:32.292 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:32.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673012 2026-02-17T12:07:32.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673012' 2026-02-17T12:07:32.293 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 42949673012 2026-02-17T12:07:32.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:32.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673013 -lt 42949673012 2026-02-17T12:07:32.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:32.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542194 2026-02-17T12:07:32.482 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:32.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:07:32.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542194 2026-02-17T12:07:32.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:32.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542194 2026-02-17T12:07:32.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542194' 2026-02-17T12:07:32.485 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 60129542194 2026-02-17T12:07:32.485 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:07:32.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542194 -lt 60129542194 2026-02-17T12:07:32.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:32.676 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776155 2026-02-17T12:07:32.676 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:32.677 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:07:32.678 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776155 2026-02-17T12:07:32.678 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:32.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776155 2026-02-17T12:07:32.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776155' 2026-02-17T12:07:32.679 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 292057776155 2026-02-17T12:07:32.680 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:07:32.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776155 -lt 292057776155 2026-02-17T12:07:32.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:32.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247855 2026-02-17T12:07:32.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:32.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:07:32.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247855 2026-02-17T12:07:32.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:32.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247855 2026-02-17T12:07:32.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247855' 2026-02-17T12:07:32.868 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 98784247855 2026-02-17T12:07:32.869 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:07:33.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247855 -lt 98784247855 2026-02-17T12:07:33.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:33.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579927 2026-02-17T12:07:33.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:33.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:07:33.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579927 2026-02-17T12:07:33.060 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:33.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579927 2026-02-17T12:07:33.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579927' 2026-02-17T12:07:33.061 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 317827579927 2026-02-17T12:07:33.061 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:07:33.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579927 -lt 317827579927 2026-02-17T12:07:33.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:33.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-365072220178 2026-02-17T12:07:33.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:33.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:07:33.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-365072220178 2026-02-17T12:07:33.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:33.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220178 2026-02-17T12:07:33.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 365072220178' 2026-02-17T12:07:33.252 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 365072220178 2026-02-17T12:07:33.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:07:33.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220178 -lt 365072220178 2026-02-17T12:07:33.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:33.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822698 2026-02-17T12:07:33.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:33.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:07:33.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822698 2026-02-17T12:07:33.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:33.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822698 2026-02-17T12:07:33.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822698' 2026-02-17T12:07:33.445 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 154618822698 2026-02-17T12:07:33.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:07:33.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822698 -lt 154618822698 2026-02-17T12:07:33.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:33.631 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659175 2026-02-17T12:07:33.631 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:33.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:07:33.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659175 2026-02-17T12:07:33.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:33.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659175 2026-02-17T12:07:33.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659175' 2026-02-17T12:07:33.634 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 176093659175 2026-02-17T12:07:33.635 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:07:33.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659175 -lt 176093659175 2026-02-17T12:07:33.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:33.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023949 2026-02-17T12:07:33.827 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:33.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:07:33.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023949 2026-02-17T12:07:33.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:33.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023949 2026-02-17T12:07:33.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023949' 2026-02-17T12:07:33.830 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.9 seq 390842023949 2026-02-17T12:07:33.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:07:34.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023950 -lt 390842023949 2026-02-17T12:07:34.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:34.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397539 2026-02-17T12:07:34.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:34.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:07:34.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397539 2026-02-17T12:07:34.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:34.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397539 2026-02-17T12:07:34.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397539' 2026-02-17T12:07:34.024 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.10 seq 210453397539 2026-02-17T12:07:34.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:07:34.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397540 -lt 210453397539 2026-02-17T12:07:34.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:34.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:34.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:34.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 == 0 2026-02-17T12:07:34.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:34.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:34.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:34.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:34.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:34.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:34.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:34.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=28 2026-02-17T12:07:34.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:34.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:34.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:34.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 28 = 28 2026-02-17T12:07:34.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:34.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:34.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:34.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:35.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:07:35.102 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 7 9 4 3 10 2026-02-17T12:07:35.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:35.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-17T12:07:35.104 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:35.105 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:35.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:07:35.297 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:35.297 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:35.297 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:35.298 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:35.298 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:35.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 7 9 4 3 10 2026-02-17T12:07:35.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:35.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:35.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:35.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-17T12:07:35.526 INFO:tasks.workunit.client.0.trial044.stderr:osd.5 is already in. 2026-02-17T12:07:35.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-17T12:07:35.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-17T12:07:35.732 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 is already in. 2026-02-17T12:07:35.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-17T12:07:35.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:35.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:35.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:35.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:35.742 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:35.742 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:35.742 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:35.742 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:35.742 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:35.824 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:36.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:36.017 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:36.017 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:07:36.017 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:8 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:36.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:36.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836536 2026-02-17T12:07:36.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836536 2026-02-17T12:07:36.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536' 2026-02-17T12:07:36.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:36.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673016 2026-02-17T12:07:36.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673016 2026-02-17T12:07:36.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016' 2026-02-17T12:07:36.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.142 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:07:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542197 2026-02-17T12:07:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542197 2026-02-17T12:07:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197' 2026-02-17T12:07:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:07:36.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776158 2026-02-17T12:07:36.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776158 2026-02-17T12:07:36.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158' 2026-02-17T12:07:36.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:07:36.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247858 2026-02-17T12:07:36.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247858 2026-02-17T12:07:36.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858' 2026-02-17T12:07:36.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:07:36.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579930 2026-02-17T12:07:36.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579930 2026-02-17T12:07:36.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930' 2026-02-17T12:07:36.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.390 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:07:36.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220181 2026-02-17T12:07:36.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220181 2026-02-17T12:07:36.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930 6-365072220181' 2026-02-17T12:07:36.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:07:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822701 2026-02-17T12:07:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822701 2026-02-17T12:07:36.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930 6-365072220181 7-154618822701' 2026-02-17T12:07:36.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.515 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:07:36.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659178 2026-02-17T12:07:36.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659178 2026-02-17T12:07:36.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930 6-365072220181 7-154618822701 8-176093659178' 2026-02-17T12:07:36.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T12:07:36.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023953 2026-02-17T12:07:36.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023953 2026-02-17T12:07:36.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930 6-365072220181 7-154618822701 8-176093659178 9-390842023953' 2026-02-17T12:07:36.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:36.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-17T12:07:36.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397542 2026-02-17T12:07:36.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397542 2026-02-17T12:07:36.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-42949673016 2-60129542197 3-292057776158 4-98784247858 5-317827579930 6-365072220181 7-154618822701 8-176093659178 9-390842023953 10-210453397542' 2026-02-17T12:07:36.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:36.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836536 2026-02-17T12:07:36.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:36.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:36.701 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836536 2026-02-17T12:07:36.701 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:36.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836536 2026-02-17T12:07:36.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836536' 2026-02-17T12:07:36.702 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 21474836536 2026-02-17T12:07:36.702 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:36.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836536 2026-02-17T12:07:36.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:37.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:37.892 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:38.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836536 2026-02-17T12:07:38.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:38.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673016 2026-02-17T12:07:38.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:38.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:38.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673016 2026-02-17T12:07:38.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:38.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673016 2026-02-17T12:07:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673016' 2026-02-17T12:07:38.085 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 42949673016 2026-02-17T12:07:38.085 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:38.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673016 -lt 42949673016 2026-02-17T12:07:38.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:38.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542197 2026-02-17T12:07:38.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:38.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:07:38.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542197 2026-02-17T12:07:38.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:38.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542197 2026-02-17T12:07:38.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542197' 2026-02-17T12:07:38.277 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 60129542197 2026-02-17T12:07:38.277 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:07:38.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542197 -lt 60129542197 2026-02-17T12:07:38.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:38.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-292057776158 2026-02-17T12:07:38.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:38.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:07:38.472 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-292057776158 2026-02-17T12:07:38.472 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:38.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776158 2026-02-17T12:07:38.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 292057776158' 2026-02-17T12:07:38.473 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 292057776158 2026-02-17T12:07:38.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:07:38.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776158 -lt 292057776158 2026-02-17T12:07:38.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:38.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247858 2026-02-17T12:07:38.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:38.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:07:38.665 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247858 2026-02-17T12:07:38.665 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:38.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247858 2026-02-17T12:07:38.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247858' 2026-02-17T12:07:38.666 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 98784247858 2026-02-17T12:07:38.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:07:38.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247858 -lt 98784247858 2026-02-17T12:07:38.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:38.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-317827579930 2026-02-17T12:07:38.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:38.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:07:38.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-317827579930 2026-02-17T12:07:38.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:38.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579930 2026-02-17T12:07:38.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 317827579930' 2026-02-17T12:07:38.860 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 317827579930 2026-02-17T12:07:38.860 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:07:39.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579930 -lt 317827579930 2026-02-17T12:07:39.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-365072220181 2026-02-17T12:07:39.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:07:39.050 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-365072220181 2026-02-17T12:07:39.050 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220181 2026-02-17T12:07:39.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 365072220181' 2026-02-17T12:07:39.051 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 365072220181 2026-02-17T12:07:39.051 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:07:39.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220181 -lt 365072220181 2026-02-17T12:07:39.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822701 2026-02-17T12:07:39.241 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:07:39.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822701 2026-02-17T12:07:39.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822701 2026-02-17T12:07:39.243 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 154618822701 2026-02-17T12:07:39.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822701' 2026-02-17T12:07:39.244 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:07:39.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822701 -lt 154618822701 2026-02-17T12:07:39.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659178 2026-02-17T12:07:39.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:07:39.434 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659178 2026-02-17T12:07:39.434 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659178 2026-02-17T12:07:39.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659178' 2026-02-17T12:07:39.436 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 176093659178 2026-02-17T12:07:39.436 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:07:39.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659179 -lt 176093659178 2026-02-17T12:07:39.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023953 2026-02-17T12:07:39.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T12:07:39.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023953 2026-02-17T12:07:39.633 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023953 2026-02-17T12:07:39.634 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.9 seq 390842023953 2026-02-17T12:07:39.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023953' 2026-02-17T12:07:39.635 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T12:07:39.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023953 -lt 390842023953 2026-02-17T12:07:39.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 10-210453397542 2026-02-17T12:07:39.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=10 2026-02-17T12:07:39.827 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 10-210453397542 2026-02-17T12:07:39.827 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397542 2026-02-17T12:07:39.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.10 seq 210453397542' 2026-02-17T12:07:39.829 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.10 seq 210453397542 2026-02-17T12:07:39.829 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-17T12:07:40.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397543 -lt 210453397542 2026-02-17T12:07:40.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:40.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:40.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:40.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 == 0 2026-02-17T12:07:40.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:40.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:40.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:40.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:40.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:40.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:40.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:40.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=28 2026-02-17T12:07:40.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:40.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:40.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:40.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 28 = 28 2026-02-17T12:07:40.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:40.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:40.721 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-17T12:07:40.721 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:07:40.721 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:40.721 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-17T12:07:40.721 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:40.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:40.914 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:10' 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 1 9 4 3 10 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-17T12:07:40.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:40.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-17T12:07:40.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=pool-jerasure 2026-02-17T12:07:40.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:07:41.135 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:07:41.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-17T12:07:41.341 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-17T12:07:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:07:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-17T12:07:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-17T12:07:41.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-17T12:07:41.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-17T12:07:41.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-17T12:07:41.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-17T12:07:41.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-17T12:07:41.846 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-lrc-a' already exists 2026-02-17T12:07:41.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:42.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-17T12:07:42.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:42.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-17T12:07:42.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:42.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:42.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:42.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:42.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:42.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-17T12:07:42.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-lrc-a 2026-02-17T12:07:42.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:42.899 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-17T12:07:42.900 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:43.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:43.094 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:43.094 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:07:43.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 9 3 2026-02-17T12:07:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-17T12:07:43.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:43.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-17T12:07:43.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-17T12:07:43.287 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 is already out. 2026-02-17T12:07:43.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:07:48.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-17T12:07:48.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-lrc-a 2026-02-17T12:07:48.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:48.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-17T12:07:48.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-17T12:07:48.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:48.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=5 2026-02-17T12:07:48.496 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:48.497 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:07:48.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 5 9 7 2026-02-17T12:07:48.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:48.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:48.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-17T12:07:48.742 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 is already in. 2026-02-17T12:07:48.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:48.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-17T12:07:48.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=pool-lrc-a 2026-02-17T12:07:48.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-17T12:07:49.039 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-lrc-a' does not exist 2026-02-17T12:07:49.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-17T12:07:49.248 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-17T12:07:49.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-17T12:07:49.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-17T12:07:49.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-17T12:07:49.802 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-lrc' already exists 2026-02-17T12:07:49.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:50.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:50.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:50.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:50.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:50.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:50.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-17T12:07:50.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-lrc 2026-02-17T12:07:50.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:50.855 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-17T12:07:50.855 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:51.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=8 2026-02-17T12:07:51.048 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:10 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:07:51.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-17T12:07:51.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-17T12:07:51.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:51.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-17T12:07:51.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-17T12:07:51.302 INFO:tasks.workunit.client.0.trial044.stderr:osd.7 is already out. 2026-02-17T12:07:51.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:07:56.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-17T12:07:56.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-lrc 2026-02-17T12:07:56.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:56.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-17T12:07:56.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-17T12:07:56.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=8 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:10 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:07:56.513 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:56.514 INFO:tasks.workunit.client.0.trial044.stderr:4' 2026-02-17T12:07:56.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-17T12:07:56.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:56.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:56.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-17T12:07:56.762 INFO:tasks.workunit.client.0.trial044.stderr:osd.7 is already in. 2026-02-17T12:07:56.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:07:56.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-17T12:07:56.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=pool-lrc 2026-02-17T12:07:56.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-17T12:07:57.060 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-lrc' does not exist 2026-02-17T12:07:57.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-17T12:07:57.293 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile profile-lrc does not exist 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-17T12:07:57.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-17T12:07:57.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-17T12:07:57.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-17T12:07:57.798 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-shec' already exists 2026-02-17T12:07:57.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-17T12:07:58.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-17T12:07:58.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:07:58.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:07:58.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-17T12:07:58.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-17T12:07:58.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-shec 2026-02-17T12:07:58.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:07:58.848 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-17T12:07:58.848 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:07:59.042 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=7 2026-02-17T12:07:59.042 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:07:59.042 INFO:tasks.workunit.client.0.trial044.stderr:1' 2026-02-17T12:07:59.042 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 7 9 1 2026-02-17T12:07:59.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-17T12:07:59.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-17T12:07:59.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-17T12:07:59.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-17T12:07:59.296 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 is already out. 2026-02-17T12:07:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-17T12:08:04.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-17T12:08:04.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-shec 2026-02-17T12:08:04.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:08:04.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-17T12:08:04.307 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-17T12:08:04.307 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:08:04.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=7 2026-02-17T12:08:04.504 INFO:tasks.workunit.client.0.trial044.stderr:9 2026-02-17T12:08:04.504 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:08:04.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 7 9 6 2026-02-17T12:08:04.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-17T12:08:04.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-17T12:08:04.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-17T12:08:04.751 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 is already in. 2026-02-17T12:08:04.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-17T12:08:04.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-17T12:08:04.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=pool-shec 2026-02-17T12:08:04.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-17T12:08:04.953 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-shec' does not exist 2026-02-17T12:08:04.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-17T12:08:05.156 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile profile-shec does not exist 2026-02-17T12:08:05.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-17T12:08:05.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=ecpool 2026-02-17T12:08:05.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-17T12:08:05.355 INFO:tasks.workunit.client.0.trial044.stderr:pool 'ecpool' does not exist 2026-02-17T12:08:05.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-17T12:08:05.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code 2026-02-17T12:08:05.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:08:05.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code KILL 2026-02-17T12:08:05.364 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:05.364 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:05.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:05.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:05.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:05.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:05.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:08:05.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:08:05.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:08:05.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:08:05.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:08:05.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:08:05.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:08:05.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:08:05.481 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:08:05.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:08:05.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code 2026-02-17T12:08:05.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:08:05.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:08:05.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.63880 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/test-erasure-code 0 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-code 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-code KILL 2026-02-17T12:08:05.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:05.547 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:08:05.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:08:05.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:08:05.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:08:05.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:08:05.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:08:05.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:08:05.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:08:05.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:08:05.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:08:05.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-code 2026-02-17T12:08:05.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:08:05.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.63880 2026-02-17T12:08:05.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.63880 2026-02-17T12:08:05.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:08:05.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:08:05.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:08:05.555 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:08:05.556 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:08:05.582 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-17T12:08:05.582 DEBUG:teuthology.orchestra.run.trial044:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-17T12:08:05.644 INFO:tasks.workunit.client.0.trial044.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/test-erasure-eio 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-17T12:08:05.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none ' 2026-02-17T12:08:05.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-17T12:08:05.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-17T12:08:05.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-17T12:08:05.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:08:05.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-17T12:08:05.650 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_object_attr_read_error 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_recovery_multiple_errors 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_recovery_multiple_objects 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_recovery_unfound 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_ec_single_recovery_error 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_bad_size_shard_0 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_bad_size_shard_1 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_subread_missing 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:08:05.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:05.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:05.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:08:05.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:08:05.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:08:05.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:08:05.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:08:05.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:08:05.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:08:05.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:08:05.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:05.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:08:05.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:08:05.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:08:05.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:08:05.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:08:05.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:08:05.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:08:05.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:08:05.661 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:08:05.661 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.661 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:08:05.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:08:05.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:05.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:08:05.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:08:05.711 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:08:05.712 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:08:05.758 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:08:05.758 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:08:05.758 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:08:05.758 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:08:05.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:08:05.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:08:05.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:08:05.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:08:05.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:05.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:05.909 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:08:05.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:08:05.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:08:05.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:08:06.024 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:08:06.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:08:07.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:08:07.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:08:07.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:08:07.033 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:08:07.074 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:08:07.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:08:07.080 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:08:05.701+0000 7fb18e21dd80 0 load: jerasure load: lrc 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_backfill_unfound: local objname=myobject 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_backfill_unfound: local lastobj=300 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:515: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:518: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:07.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:519: TEST_ec_backfill_unfound: setup_osds 5 2026-02-17T12:08:07.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-17T12:08:07.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:08:07.083 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-17T12:08:07.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:07.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:07.088 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:07.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=eec558da-d2ec-4cc4-a065-7948720cdbba 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 eec558da-d2ec-4cc4-a065-7948720cdbba' 2026-02-17T12:08:07.090 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 eec558da-d2ec-4cc4-a065-7948720cdbba 2026-02-17T12:08:07.091 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:07.099 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAnWpRpKXfQBRAAxfZFNSIxzjD0hkTwDZnPLg== 2026-02-17T12:08:07.099 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAnWpRpKXfQBRAAxfZFNSIxzjD0hkTwDZnPLg=="}' 2026-02-17T12:08:07.099 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new eec558da-d2ec-4cc4-a065-7948720cdbba -i td/test-erasure-eio/0/new.json 2026-02-17T12:08:07.205 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:07.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:08:07.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAnWpRpKXfQBRAAxfZFNSIxzjD0hkTwDZnPLg== --osd-uuid eec558da-d2ec-4cc4-a065-7948720cdbba 2026-02-17T12:08:07.232 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:07.230+0000 7f1910623780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.233 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:07.231+0000 7f1910623780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.234 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:07.232+0000 7f1910623780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:07.233+0000 7f1910623780 -1 bdev(0x558f086ffc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:07.233+0000 7f1910623780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:08:09.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:08:09.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:09.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:08:09.606 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:08:09.606 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:09.865 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:08:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:08:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:08:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:09.885 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:09.883+0000 7ff6ddc18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:09.886 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:09.885+0000 7ff6ddc18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:09.887 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:09.886+0000 7ff6ddc18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:10.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:10.057 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:10.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:10.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:10.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:11.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:11.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:11.248 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:11.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:11.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:11.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:11.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:11.468 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:11.466+0000 7ff6ddc18780 -1 Falling back to public interface 2026-02-17T12:08:12.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:12.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:12.439 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:12.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:12.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:12.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:12.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:13.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:13.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:13.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:13.633 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:13.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:13.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:13.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:14.415 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:14.413+0000 7ff6ddc18780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:08:14.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:14.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:14.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:14.824 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:14.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:14.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:15.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:16.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:16.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:16.019 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:08:16.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:16.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:16.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:16.211 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4005362135,v1:127.0.0.1:6803/4005362135] [v2:127.0.0.1:6804/4005362135,v1:127.0.0.1:6805/4005362135] exists,up eec558da-d2ec-4cc4-a065-7948720cdbba 2026-02-17T12:08:16.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:16.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:16.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:16.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:08:16.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:08:16.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:16.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:08:16.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:16.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3f85e399-9b9a-4072-9a8a-b77dfbc13c69 2026-02-17T12:08:16.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3f85e399-9b9a-4072-9a8a-b77dfbc13c69' 2026-02-17T12:08:16.219 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 3f85e399-9b9a-4072-9a8a-b77dfbc13c69 2026-02-17T12:08:16.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:16.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAwWpRpqrCADRAAE5U2Us7MeGIhdHuFYoI20w== 2026-02-17T12:08:16.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAwWpRpqrCADRAAE5U2Us7MeGIhdHuFYoI20w=="}' 2026-02-17T12:08:16.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3f85e399-9b9a-4072-9a8a-b77dfbc13c69 -i td/test-erasure-eio/1/new.json 2026-02-17T12:08:16.421 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:16.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:08:16.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwWpRpqrCADRAAE5U2Us7MeGIhdHuFYoI20w== --osd-uuid 3f85e399-9b9a-4072-9a8a-b77dfbc13c69 2026-02-17T12:08:16.448 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:16.447+0000 7f54c32e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:16.449 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:16.448+0000 7f54c32e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:16.450 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:16.449+0000 7f54c32e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:16.451 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:16.450+0000 7f54c32e7780 -1 bdev(0x55bab730bc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:16.451 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:16.450+0000 7f54c32e7780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:08:18.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:08:18.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:18.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:08:18.562 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:08:18.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:18.841 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:08:18.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:08:18.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:08:18.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:18.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:18.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:18.860 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:18.859+0000 7f6396b95780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:18.861 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:18.860+0000 7f6396b95780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:18.862 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:18.861+0000 7f6396b95780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:19.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:19.037 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:19.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:20.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:20.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:20.451 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:20.449+0000 7f6396b95780 -1 Falling back to public interface 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:21.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:22.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:22.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:22.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:22.609 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:22.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:22.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:22.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:23.401 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:23.400+0000 7f6396b95780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:08:23.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:23.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:23.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:23.800 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:23.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:23.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:23.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:24.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:24.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:24.990 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:08:24.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:24.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:24.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:25.180 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3865764537,v1:127.0.0.1:6811/3865764537] [v2:127.0.0.1:6812/3865764537,v1:127.0.0.1:6813/3865764537] exists,up 3f85e399-9b9a-4072-9a8a-b77dfbc13c69 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:08:25.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:08:25.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:25.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:25.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f' 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:08:25.189 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:25.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA5WpRp9VuvCxAAl5UDAxC2q9024qfEBvANwA== 2026-02-17T12:08:25.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA5WpRp9VuvCxAAl5UDAxC2q9024qfEBvANwA=="}' 2026-02-17T12:08:25.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f -i td/test-erasure-eio/2/new.json 2026-02-17T12:08:25.388 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:25.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:08:25.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5WpRp9VuvCxAAl5UDAxC2q9024qfEBvANwA== --osd-uuid 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:08:25.415 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:25.414+0000 7f0a476d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:25.417 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:25.416+0000 7f0a476d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:25.417 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:25.417+0000 7f0a476d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:25.418 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:25.417+0000 7f0a476d5780 -1 bdev(0x55a028b87c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:25.418 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:25.417+0000 7f0a476d5780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:08:27.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:08:27.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:27.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:08:27.538 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:08:27.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:27.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:08:27.799 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:08:27.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:08:27.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:27.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:27.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:27.817 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:27.816+0000 7fed0482f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:27.818 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:27.817+0000 7fed0482f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:27.819 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:27.818+0000 7fed0482f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:27.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:08:27.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:27.990 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:27.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:27.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:08:28.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:29.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:29.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:29.181 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:29.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:29.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:29.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:08:29.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:29.411 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:29.409+0000 7fed0482f780 -1 Falling back to public interface 2026-02-17T12:08:30.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:30.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:30.373 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:30.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:30.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:30.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:08:30.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:31.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:08:31.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:32.113 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:32.112+0000 7fed0482f780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:08:32.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:32.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:32.770 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:32.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:32.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:32.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:08:32.965 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/108612143,v1:127.0.0.1:6819/108612143] [v2:127.0.0.1:6820/108612143,v1:127.0.0.1:6821/108612143] exists,up 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:08:32.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:08:32.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:32.971 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:32.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6457e6a7-c033-4e77-a6b4-212cdaff0722 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 6457e6a7-c033-4e77-a6b4-212cdaff0722' 2026-02-17T12:08:32.973 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 6457e6a7-c033-4e77-a6b4-212cdaff0722 2026-02-17T12:08:32.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:32.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBAWpRpGF52OhAAQBDRfHHH6LXqMiFPfJDMqw== 2026-02-17T12:08:32.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBAWpRpGF52OhAAQBDRfHHH6LXqMiFPfJDMqw=="}' 2026-02-17T12:08:32.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6457e6a7-c033-4e77-a6b4-212cdaff0722 -i td/test-erasure-eio/3/new.json 2026-02-17T12:08:33.181 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:33.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:08:33.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAWpRpGF52OhAAQBDRfHHH6LXqMiFPfJDMqw== --osd-uuid 6457e6a7-c033-4e77-a6b4-212cdaff0722 2026-02-17T12:08:33.209 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:33.208+0000 7f973e7ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:33.210 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:33.209+0000 7f973e7ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:33.211 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:33.210+0000 7f973e7ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:33.212 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:33.211+0000 7f973e7ab780 -1 bdev(0x55c1e032bc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:33.212 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:33.211+0000 7f973e7ab780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:08:35.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:08:35.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:35.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:08:35.574 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:08:35.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:35.829 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:08:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:08:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:08:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:35.848 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:35.846+0000 7f117540f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:35.849 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:35.848+0000 7f117540f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:35.850 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:35.849+0000 7f117540f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:36.020 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:36.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:36.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:36.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:36.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:37.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:37.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:37.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:37.213 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:37.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:37.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:37.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:37.680 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:37.679+0000 7f117540f780 -1 Falling back to public interface 2026-02-17T12:08:38.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:38.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:38.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:38.404 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:38.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:38.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:38.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:39.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:39.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:39.596 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:39.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:39.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:39.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:39.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:40.391 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:40.389+0000 7f117540f780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:08:40.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:40.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:40.786 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:40.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:40.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:40.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:40.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:41.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:41.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:41.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:41.982 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:08:41.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:41.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:08:42.171 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4017407891,v1:127.0.0.1:6827/4017407891] [v2:127.0.0.1:6828/4017407891,v1:127.0.0.1:6829/4017407891] exists,up 6457e6a7-c033-4e77-a6b4-212cdaff0722 2026-02-17T12:08:42.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:42.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:42.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:08:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:08:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:08:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:42.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:08:42.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:08:42.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:42.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:42.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:42.183 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:42.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:42.185 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:42.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:08:42.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:08:42.186 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:42.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=dabf9236-de19-4cf1-9d1f-aeee06639fa9 2026-02-17T12:08:42.187 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 dabf9236-de19-4cf1-9d1f-aeee06639fa9 2026-02-17T12:08:42.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 dabf9236-de19-4cf1-9d1f-aeee06639fa9' 2026-02-17T12:08:42.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:42.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBKWpRpSz8oCxAAME5KMuuOFCLSLnVJzG348w== 2026-02-17T12:08:42.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBKWpRpSz8oCxAAME5KMuuOFCLSLnVJzG348w=="}' 2026-02-17T12:08:42.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new dabf9236-de19-4cf1-9d1f-aeee06639fa9 -i td/test-erasure-eio/4/new.json 2026-02-17T12:08:42.381 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:42.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:08:42.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKWpRpSz8oCxAAME5KMuuOFCLSLnVJzG348w== --osd-uuid dabf9236-de19-4cf1-9d1f-aeee06639fa9 2026-02-17T12:08:42.410 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:42.408+0000 7fab07e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:42.411 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:42.409+0000 7fab07e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:42.412 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:42.410+0000 7fab07e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:42.412 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:42.410+0000 7fab07e0c780 -1 bdev(0x5584670bfc00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:42.412 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:42.410+0000 7fab07e0c780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:08:44.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:08:44.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:44.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:08:44.527 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:08:44.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:44.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:08:44.784 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:08:44.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:08:44.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:44.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:44.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:44.803 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:44.801+0000 7f936fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:44.804 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:44.802+0000 7f936fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:44.805 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:44.803+0000 7f936fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:44.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:44.981 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:08:44.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:44.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:45.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:46.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:46.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:46.174 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:08:46.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:46.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:46.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:46.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:46.394 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:46.391+0000 7f936fe0f780 -1 Falling back to public interface 2026-02-17T12:08:47.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:47.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:47.370 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:08:47.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:47.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:47.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:47.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:48.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:48.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:48.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:48.567 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:08:48.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:48.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:48.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:49.352 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:08:49.351+0000 7f936fe0f780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:08:49.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:49.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:49.765 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:08:49.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:49.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:49.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:49.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:50.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:50.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:50.956 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:08:50.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:50.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:50.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:08:51.149 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2462213402,v1:127.0.0.1:6835/2462213402] [v2:127.0.0.1:6836/2462213402,v1:127.0.0.1:6837/2462213402] exists,up dabf9236-de19-4cf1-9d1f-aeee06639fa9 2026-02-17T12:08:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:08:51.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:08:51.195 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:08:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:08:51.203 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:08:11.469+0000 7ff6ddc18780 0 load: jerasure load: lrc 2026-02-17T12:08:51.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:08:51.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-17T12:08:51.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:08:51.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:08:51.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:08:51.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:08:51.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:08:51.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:08:51.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:08:51.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:08:51.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:08:51.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:08:51.748 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:08:51.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:08:52.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:08:52.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:08:52.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:08:52.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:08:52.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:08:52.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:08:52.760 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:08:52.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:08:52.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:08:52.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:08:52.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:08:52.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:08:52.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:08:52.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:08:52.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:08:52.842 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:08:53.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:4' 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:53.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:08:53.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803786 2026-02-17T12:08:53.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803786 2026-02-17T12:08:53.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786' 2026-02-17T12:08:53.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:53.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:08:53.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607560 2026-02-17T12:08:53.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607560 2026-02-17T12:08:53.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607560' 2026-02-17T12:08:53.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:53.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:08:53.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411334 2026-02-17T12:08:53.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411334 2026-02-17T12:08:53.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607560 2-77309411334' 2026-02-17T12:08:53.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:53.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:08:53.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149701 2026-02-17T12:08:53.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149701 2026-02-17T12:08:53.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607560 2-77309411334 3-111669149701' 2026-02-17T12:08:53.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:53.277 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:08:53.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855363 2026-02-17T12:08:53.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855363 2026-02-17T12:08:53.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607560 2-77309411334 3-111669149701 4-150323855363' 2026-02-17T12:08:53.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:53.342 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803786 2026-02-17T12:08:53.342 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:53.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:08:53.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803786 2026-02-17T12:08:53.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:53.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803786 2026-02-17T12:08:53.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-17T12:08:53.344 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803786 2026-02-17T12:08:53.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:53.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-17T12:08:53.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:08:54.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:08:54.537 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:54.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-17T12:08:54.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:54.733 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607560 2026-02-17T12:08:54.733 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:54.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:08:54.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607560 2026-02-17T12:08:54.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:54.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607560 2026-02-17T12:08:54.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-17T12:08:54.736 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607560 2026-02-17T12:08:54.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:08:54.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-17T12:08:54.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:54.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411334 2026-02-17T12:08:54.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:54.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:08:54.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411334 2026-02-17T12:08:54.937 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:54.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411334 2026-02-17T12:08:54.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-17T12:08:54.938 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411334 2026-02-17T12:08:54.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:08:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411335 -lt 77309411334 2026-02-17T12:08:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:55.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149701 2026-02-17T12:08:55.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:08:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149701 2026-02-17T12:08:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149701 2026-02-17T12:08:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-17T12:08:55.131 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149701 2026-02-17T12:08:55.131 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:08:55.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-17T12:08:55.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:55.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855363 2026-02-17T12:08:55.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:55.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:08:55.319 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855363 2026-02-17T12:08:55.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:55.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855363 2026-02-17T12:08:55.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855363' 2026-02-17T12:08:55.321 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855363 2026-02-17T12:08:55.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:08:55.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-17T12:08:55.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:08:55.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:08:55.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:08:55.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:08:55.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:08:55.948 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:08:55.948 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:08:55.948 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:08:56.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:08:56.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:08:56.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:08:56.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:525: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:52.094667+0000 0'0 42:12 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-17T12:08:51.696178+0000 0'0 2026-02-17T12:08:51.696178+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:29:52.006943+0000 0 0 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361385+0000 0'0 43:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T14:19:24.273058+0000 0 0 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361662+0000 0'0 43:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:24:28.001368+0000 0 0 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.444008+0000 0'0 43:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:23:30.299365+0000 0 0 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:33.231673+0000 0'0 43:65 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T18:37:24.469828+0000 0 0 2026-02-17T12:08:56.387 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:08:56.388 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:08:56.388 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:08:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:527: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:08:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:08:56.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:08:56.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:56.414 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:528: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-17T12:08:56.414 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=pool-jerasure 2026-02-17T12:08:56.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=myobject 2026-02-17T12:08:56.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:08:56.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:08:56.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:528: TEST_ec_backfill_unfound: local primary=3 2026-02-17T12:08:56.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:530: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-17T12:08:56.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:08:56.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:08:56.610 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:08:56.610 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:08:56.804 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:530: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:530: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local last_osd=2 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:08:56.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:56.806 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:56.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:56.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:56.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:58.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:58.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-17T12:08:58.306 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already down. 2026-02-17T12:08:58.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:534: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-17T12:08:58.559 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already out. 2026-02-17T12:08:58.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:536: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-17T12:08:58.748 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:08:58.748 INFO:tasks.workunit.client.0.trial044.stdout:2.0 1 0 0 0 0 4096 0 0 1 0 1 active+clean 2026-02-17T12:08:52.094667+0000 43'1 43:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-17T12:08:51.696178+0000 0'0 2026-02-17T12:08:51.696178+0000 0 0 periodic scrub scheduled @ 2026-02-18T13:26:01.161232+0000 0 0 2026-02-17T12:08:58.748 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361385+0000 0'0 43:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T14:19:24.273058+0000 0 0 2026-02-17T12:08:58.748 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361662+0000 0'0 43:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:24:28.001368+0000 0 0 2026-02-17T12:08:58.749 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.444008+0000 0'0 43:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:23:30.299365+0000 0 0 2026-02-17T12:08:58.749 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:33.231673+0000 0'0 43:65 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T18:37:24.469828+0000 0 0 2026-02-17T12:08:58.749 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:08:58.749 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:08:58.749 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:08:58.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-17T12:08:58.756 INFO:tasks.workunit.client.0.trial044.stderr:4+0 records in 2026-02-17T12:08:58.756 INFO:tasks.workunit.client.0.trial044.stderr:4+0 records out 2026-02-17T12:08:58.756 INFO:tasks.workunit.client.0.trial044.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.5961e-05 s, 53.9 MB/s 2026-02-17T12:08:58.756 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: seq 1 300 2026-02-17T12:08:58.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:58.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:58.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-17T12:08:59.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:08:59.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.715 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.715 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:00.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:00.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.922 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.922 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:01.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:01.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.331 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.331 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.498 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:02.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:02.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:03.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:03.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:04.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:541: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:544: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:09:04.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj250 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj250 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj250 2026-02-17T12:09:04.320 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-17T12:09:04.321 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:2147483647' 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2147483647 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:09:04.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:09:04.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:04.515 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:09:04.516 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:09:04.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:09:04.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:09:04.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:09:04.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:09:04.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj250 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj250 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj250 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-17T12:09:04.617 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr:2147483647' 2026-02-17T12:09:04.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2147483647 2026-02-17T12:09:04.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-17T12:09:04.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:09:04.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:09:04.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:09:04.806 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/1/type 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:09:04.808 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:09:04.809 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:04.809 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:04.809 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:04.809 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:09:04.809 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:09:04.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:09:04.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:09:04.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-17T12:09:04.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-17T12:09:04.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:09:04.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:09:04.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:04.906 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:09:04.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:09:04.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:09:04.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:09:04.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:09:04.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:09:04.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:09:04.923 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:04.923+0000 7f6d26a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:04.925 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:04.924+0000 7f6d26a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:04.926 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:04.925+0000 7f6d26a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:05.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:09:05.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:05.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:09:05.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:05.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:05.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:05.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:05.104 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:09:05.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:05.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:05.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:06.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:06.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:06.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:06.290 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:09:06.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:06.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:06.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:06.509 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:06.507+0000 7f6d26a1a780 -1 Falling back to public interface 2026-02-17T12:09:07.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:07.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:07.481 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:09:07.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:07.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:07.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:07.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:08.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:08.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:08.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:08.671 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:09:08.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:08.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:08.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:09.468 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:09.467+0000 7f6d26a1a780 -1 osd.2 43 log_to_monitors true 2026-02-17T12:09:09.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:09.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:09.863 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:09:09.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:09.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:09.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:10.050 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/4149407674,v1:127.0.0.1:6819/4149407674] [v2:127.0.0.1:6820/4149407674,v1:127.0.0.1:6821/4149407674] exists,up 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-17T12:09:10.337 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already in. 2026-02-17T12:09:10.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: sleep 15 2026-02-17T12:09:25.347 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: seq 1 240 2026-02-17T12:09:25.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-17T12:09:25.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: get_state 2.0 2026-02-17T12:09:25.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-17T12:09:25.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-17T12:09:25.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-17T12:09:25.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-17T12:09:25.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-17T12:09:25.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-17T12:09:25.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-17T12:09:25.539 INFO:tasks.workunit.client.0.trial044.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-17T12:09:25.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:555: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-17T12:09:25.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: break 2026-02-17T12:09:25.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:562: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-17T12:09:25.718 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:09:25.718 INFO:tasks.workunit.client.0.trial044.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-17T12:09:10.940426+0000 47'301 53:894 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-17T12:08:51.696178+0000 0'0 2026-02-17T12:08:51.696178+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:09:10.390298+0000 0'0 53:56 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T17:14:23.048746+0000 0 0 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361662+0000 0'0 53:113 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:24:28.001368+0000 0 0 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.444008+0000 0'0 53:41 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:23:30.299365+0000 0 0 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:09:10.390292+0000 0'0 53:100 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T23:23:50.337995+0000 0 0 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:09:25.719 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:09:25.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:563: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:09:25.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:25.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:25.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:25.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:25.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:27.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:27.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: sleep 5 2026-02-17T12:09:32.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:566: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-17T12:09:32.214 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:09:32.214 INFO:tasks.workunit.client.0.trial044.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-17T12:09:25.889430+0000 47'301 55:906 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-17T12:08:51.696178+0000 0'0 2026-02-17T12:08:51.696178+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:09:32.214 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T12:09:25.887701+0000 0'0 55:63 [4,1] 4 [4,1] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.361662+0000 0'0 55:117 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:24:28.001368+0000 0 0 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:08:50.444008+0000 0'0 55:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:23:30.299365+0000 0 0 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T12:09:25.887359+0000 0'0 55:107 [1,0] 1 [1,0] 1 0'0 2026-02-17T12:08:06.005708+0000 0'0 2026-02-17T12:08:06.005708+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:09:32.215 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:09:32.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout:{ 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: "num_missing": 1, 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: "num_unfound": 1, 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: "objects": [ 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: "oid": { 2026-02-17T12:09:32.283 INFO:tasks.workunit.client.0.trial044.stdout: "oid": "obj250", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "key": "", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "snapid": -2, 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "hash": 2249616407, 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "max": 0, 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "pool": 2, 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "namespace": "" 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "need": "47'251", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "have": "0'0", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "flags": "none", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-17T12:09:32.284 INFO:tasks.workunit.client.0.trial044.stdout: "locations": [ 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: ] 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "state": "NotRecovering", 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "available_might_have_unfound": true, 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "might_have_unfound": [ 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "2(4)", 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: "status": "osd is down" 2026-02-17T12:09:32.285 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.286 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.286 INFO:tasks.workunit.client.0.trial044.stdout: "more": false 2026-02-17T12:09:32.286 INFO:tasks.workunit.client.0.trial044.stdout:} 2026-02-17T12:09:32.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout:{ 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: "snap_trimq": "[]", 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: "snap_trimq_len": 0, 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: "epoch": 55, 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:09:32.347 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.348 INFO:tasks.workunit.client.0.trial044.stdout: "acting_recovery_backfill": [ 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "info": { 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s0", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "0", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "47'301", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "47'250", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "47'200", 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 301, 2026-02-17T12:09:32.349 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 41, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 41, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 54, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 41, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 54, 2026-02-17T12:09:32.350 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 54, 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 41, 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "version": "47'301", 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 907, 2026-02-17T12:09:32.351 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 55, 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:09:25.889430+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:09:25.889430+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:09:25.889430+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:09:25.889430+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:08:56.412141+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:09:25.888018+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:09:25.888018+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:09:25.889430+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:09:25.886745+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:09:25.886572+0000", 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 54, 2026-02-17T12:09:32.352 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "47'200", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "47'200", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "created": 41, 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:09:32.353 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 200, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:09:32.354 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 1232896, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 301, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 1505, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 1, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 1, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 303, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 1, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 301, 2026-02-17T12:09:32.355 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 301, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 1204, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:09:32.356 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:09:32.357 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.358 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:09:32.359 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 300 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),4(2)", 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 1 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.360 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "peer_info": [ 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "0(3)", 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s3", 2026-02-17T12:09:32.361 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "3", 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "47'301", 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "47'301", 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "47'200", 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 301, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 41, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 41, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 54, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 41, 2026-02-17T12:09:32.362 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 54, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 54, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 41, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:09:32.363 INFO:tasks.workunit.client.0.trial044.stdout: "version": "47'301", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 897, 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 54, 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:08:56.412141+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:08:57.000611+0000", 2026-02-17T12:09:32.364 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:08:57.000545+0000", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 54, 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "47'200", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "47'200", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "created": 41, 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.365 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 200, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:09:32.366 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T13:42:57.911501+0000", 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 1232896, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 301, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 1505, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 0, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 301, 2026-02-17T12:09:32.367 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 301, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 301, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 1204, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.370 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.371 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 301 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:09:32.372 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "1(1)", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s1", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "1", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "47'301", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "47'301", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "47'200", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 301, 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 41, 2026-02-17T12:09:32.373 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 41, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 54, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 41, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 54, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 54, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 41, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.374 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "version": "47'301", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 329, 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 47, 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:08:56.412141+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:08:57.000611+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:08:57.000545+0000", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 54, 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "47'200", 2026-02-17T12:09:32.375 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "47'200", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "created": 41, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 200, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:09:32.376 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T13:42:57.911501+0000", 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 1232896, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 301, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 1505, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 1, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 301, 2026-02-17T12:09:32.377 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 301, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 301, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 1204, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:09:32.378 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.379 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:09:32.380 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 301 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:09:32.381 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "4(2)", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s2", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "2", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "47'301", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "47'301", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "47'200", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 301, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 41, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 41, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 54, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 41, 2026-02-17T12:09:32.382 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 54, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 54, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 41, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "version": "47'301", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 329, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 47, 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.383 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:08:56.412141+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:08:57.002101+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:09:04.316121+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:08:57.000611+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:08:57.000545+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 54, 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "47'200", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "47'200", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "created": 41, 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 42, 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:08:51.696178+0000", 2026-02-17T12:09:32.384 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 200, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T13:42:57.911501+0000", 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 1232896, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 301, 2026-02-17T12:09:32.385 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 1505, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 301, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 301, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 301, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 1204, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:09:32.386 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.387 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: 2147483647 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 301 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:09:32.388 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 55, 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_state": [ 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "name": "Started/Primary/Active", 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "enter_time": "2026-02-17T12:09:25.886605+0000", 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: "might_have_unfound": [ 2026-02-17T12:09:32.389 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "0(3)", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "1(1)", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "2(4)", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "status": "osd is down" 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "4(2)", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_progress": { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "backfill_targets": [], 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "waiting_on_backfill": [], 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill_started": "MIN", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "backfill_info": { 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "begin": "MIN", 2026-02-17T12:09:32.390 INFO:tasks.workunit.client.0.trial044.stdout: "end": "MIN", 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "objects": [] 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "peer_backfill_info": [], 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "backfills_in_flight": [], 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "recovering": [], 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "pg_backend": { 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_ops": [], 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "read_ops": [] 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "name": "Started", 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "enter_time": "2026-02-17T12:09:25.783256+0000" 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "scrubber": { 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "active": false, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "must_scrub": false, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "must_deep_scrub": false, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "must_repair": false, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "need_auto": false, 2026-02-17T12:09:32.391 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_reg_stamp": "2026-02-18T13:42:57.911501+0000", 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stdout: "schedule": "no scrub is scheduled" 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stdout: "agent_state": {} 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stdout:} 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:09:32.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: grep -q obj250 2026-02-17T12:09:32.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:09:32.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-17T12:09:32.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: check=true 2026-02-17T12:09:32.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: test true == true 2026-02-17T12:09:32.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:09:32.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-17T12:09:32.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-17T12:09:32.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: check='osd is down' 2026-02-17T12:09:32.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-17T12:09:32.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:578: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:09:32.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:578: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:578: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:578: TEST_ec_backfill_unfound: check='2(4)' 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:579: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:581: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:09:32.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:32.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:32.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:09:32.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:09:32.612 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:09:32.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:09:32.624 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:32.622+0000 7f0f254e9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:32.625 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:32.624+0000 7f0f254e9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:32.626 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:32.625+0000 7f0f254e9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:32.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:09:32.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:32.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:09:32.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:32.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:32.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:32.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:32.798 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:09:32.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:32.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:32.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:33.959 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:33.957+0000 7f0f254e9780 -1 Falling back to public interface 2026-02-17T12:09:33.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:33.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:33.989 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:09:33.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:33.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:33.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:34.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:35.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:35.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:35.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:35.178 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:09:35.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:35.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:35.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:36.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:36.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:36.369 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:09:36.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:36.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:36.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:36.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:36.671 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:36.670+0000 7f0f254e9780 -1 osd.2 53 log_to_monitors true 2026-02-17T12:09:37.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:37.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:37.556 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:09:37.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:37.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:37.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:37.746 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 57 up_thru 20 down_at 54 last_clean_interval [49,53) [v2:127.0.0.1:6818/3570932665,v1:127.0.0.1:6819/3570932665] [v2:127.0.0.1:6820/3570932665,v1:127.0.0.1:6821/3570932665] exists,up 66f00c5a-b6fa-408e-bd8d-30849cb7fd9f 2026-02-17T12:09:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:37.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:37.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:37.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-17T12:09:42.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:585: TEST_ec_backfill_unfound: test 124 = 124 2026-02-17T12:09:42.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-17T12:09:42.812 INFO:tasks.workunit.client.0.trial044.stderr:pg has no unfound objects 2026-02-17T12:09:42.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:589: TEST_ec_backfill_unfound: wait_for_clean 2026-02-17T12:09:42.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:09:42.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:09:42.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:09:42.818 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:09:42.819 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:09:42.819 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:09:42.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:09:42.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:09:42.820 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:09:42.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:4' 2026-02-17T12:09:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:09:43.098 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:43.098 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:09:43.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803798 2026-02-17T12:09:43.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803798 2026-02-17T12:09:43.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803798' 2026-02-17T12:09:43.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:43.161 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:09:43.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607572 2026-02-17T12:09:43.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607572 2026-02-17T12:09:43.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803798 1-51539607572' 2026-02-17T12:09:43.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:43.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:09:43.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135876 2026-02-17T12:09:43.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135876 2026-02-17T12:09:43.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803798 1-51539607572 2-244813135876' 2026-02-17T12:09:43.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:09:43.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149713 2026-02-17T12:09:43.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149713 2026-02-17T12:09:43.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803798 1-51539607572 2-244813135876 3-111669149713' 2026-02-17T12:09:43.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:43.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:09:43.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855375 2026-02-17T12:09:43.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855375 2026-02-17T12:09:43.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803798 1-51539607572 2-244813135876 3-111669149713 4-150323855375' 2026-02-17T12:09:43.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:43.414 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803798 2026-02-17T12:09:43.414 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:43.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:09:43.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803798 2026-02-17T12:09:43.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:43.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803798 2026-02-17T12:09:43.416 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803798 2026-02-17T12:09:43.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-17T12:09:43.417 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:43.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-17T12:09:43.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:44.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:09:44.606 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:44.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803798 -lt 25769803798 2026-02-17T12:09:44.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:44.804 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607572 2026-02-17T12:09:44.804 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:44.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:09:44.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607572 2026-02-17T12:09:44.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:44.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607572 2026-02-17T12:09:44.806 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607572 2026-02-17T12:09:44.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607572' 2026-02-17T12:09:44.807 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:09:45.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607572 -lt 51539607572 2026-02-17T12:09:45.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:45.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-244813135876 2026-02-17T12:09:45.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:45.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:09:45.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-244813135876 2026-02-17T12:09:45.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:45.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135876 2026-02-17T12:09:45.003 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 244813135876 2026-02-17T12:09:45.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 244813135876' 2026-02-17T12:09:45.003 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:09:45.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135876 -lt 244813135876 2026-02-17T12:09:45.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:45.197 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149713 2026-02-17T12:09:45.197 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:45.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:09:45.198 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149713 2026-02-17T12:09:45.198 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:45.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149713 2026-02-17T12:09:45.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149713' 2026-02-17T12:09:45.199 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149713 2026-02-17T12:09:45.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:09:45.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149713 -lt 111669149713 2026-02-17T12:09:45.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:45.394 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855375 2026-02-17T12:09:45.394 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:45.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:09:45.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855375 2026-02-17T12:09:45.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:45.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855375 2026-02-17T12:09:45.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855375' 2026-02-17T12:09:45.397 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855375 2026-02-17T12:09:45.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:09:45.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855375 -lt 150323855375 2026-02-17T12:09:45.591 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:09:45.591 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:45.591 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:45.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:45.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:46.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:09:46.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:46.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:46.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:46.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:09:46.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:09:46.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:09:46.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: seq 1 300 2026-02-17T12:09:46.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-17T12:09:46.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-17T12:09:46.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-17T12:09:46.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-17T12:09:46.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-17T12:09:46.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-17T12:09:46.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-17T12:09:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-17T12:09:46.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-17T12:09:46.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-17T12:09:46.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-17T12:09:46.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-17T12:09:46.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-17T12:09:46.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-17T12:09:46.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-17T12:09:46.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-17T12:09:46.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-17T12:09:46.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-17T12:09:46.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-17T12:09:46.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-17T12:09:46.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-17T12:09:46.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-17T12:09:46.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-17T12:09:46.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-17T12:09:46.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-17T12:09:46.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-17T12:09:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-17T12:09:46.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-17T12:09:46.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-17T12:09:46.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-17T12:09:46.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-17T12:09:46.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-17T12:09:46.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-17T12:09:46.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-17T12:09:46.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-17T12:09:46.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-17T12:09:46.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.948 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-17T12:09:46.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-17T12:09:46.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-17T12:09:46.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:46.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:46.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-17T12:09:46.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-17T12:09:47.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-17T12:09:47.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-17T12:09:47.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-17T12:09:47.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-17T12:09:47.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-17T12:09:47.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-17T12:09:47.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-17T12:09:47.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-17T12:09:47.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-17T12:09:47.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-17T12:09:47.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-17T12:09:47.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-17T12:09:47.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-17T12:09:47.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-17T12:09:47.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-17T12:09:47.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-17T12:09:47.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-17T12:09:47.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-17T12:09:47.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-17T12:09:47.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-17T12:09:47.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-17T12:09:47.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-17T12:09:47.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-17T12:09:47.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-17T12:09:47.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-17T12:09:47.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-17T12:09:47.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-17T12:09:47.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-17T12:09:47.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-17T12:09:47.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-17T12:09:47.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-17T12:09:47.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-17T12:09:47.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-17T12:09:47.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-17T12:09:47.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-17T12:09:47.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-17T12:09:47.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-17T12:09:47.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-17T12:09:47.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-17T12:09:47.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-17T12:09:47.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-17T12:09:47.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-17T12:09:47.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-17T12:09:47.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-17T12:09:47.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-17T12:09:47.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-17T12:09:47.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-17T12:09:47.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-17T12:09:47.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-17T12:09:47.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-17T12:09:47.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-17T12:09:47.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-17T12:09:47.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:47.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-17T12:09:47.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-17T12:09:47.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:47.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-17T12:09:48.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-17T12:09:48.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-17T12:09:48.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-17T12:09:48.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-17T12:09:48.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-17T12:09:48.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-17T12:09:48.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-17T12:09:48.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-17T12:09:48.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-17T12:09:48.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-17T12:09:48.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-17T12:09:48.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-17T12:09:48.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-17T12:09:48.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-17T12:09:48.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-17T12:09:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-17T12:09:48.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-17T12:09:48.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-17T12:09:48.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-17T12:09:48.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-17T12:09:48.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-17T12:09:48.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-17T12:09:48.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-17T12:09:48.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-17T12:09:48.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-17T12:09:48.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-17T12:09:48.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.498 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-17T12:09:48.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-17T12:09:48.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-17T12:09:48.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-17T12:09:48.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-17T12:09:48.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-17T12:09:48.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-17T12:09:48.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-17T12:09:48.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-17T12:09:48.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-17T12:09:48.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-17T12:09:48.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-17T12:09:48.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-17T12:09:48.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-17T12:09:48.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-17T12:09:48.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-17T12:09:48.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-17T12:09:48.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-17T12:09:48.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-17T12:09:48.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-17T12:09:48.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-17T12:09:48.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-17T12:09:48.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-17T12:09:48.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.922 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-17T12:09:48.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-17T12:09:48.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-17T12:09:48.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-17T12:09:48.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-17T12:09:48.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:48.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:48.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-17T12:09:48.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-17T12:09:49.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-17T12:09:49.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-17T12:09:49.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-17T12:09:49.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-17T12:09:49.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-17T12:09:49.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-17T12:09:49.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-17T12:09:49.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-17T12:09:49.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-17T12:09:49.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-17T12:09:49.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-17T12:09:49.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-17T12:09:49.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-17T12:09:49.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-17T12:09:49.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-17T12:09:49.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-17T12:09:49.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-17T12:09:49.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-17T12:09:49.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-17T12:09:49.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-17T12:09:49.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-17T12:09:49.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-17T12:09:49.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-17T12:09:49.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-17T12:09:49.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-17T12:09:49.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-17T12:09:49.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-17T12:09:49.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-17T12:09:49.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-17T12:09:49.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-17T12:09:49.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-17T12:09:49.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-17T12:09:49.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-17T12:09:49.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-17T12:09:49.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-17T12:09:49.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-17T12:09:49.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-17T12:09:49.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-17T12:09:49.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-17T12:09:49.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-17T12:09:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-17T12:09:49.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-17T12:09:49.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-17T12:09:49.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-17T12:09:49.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-17T12:09:49.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-17T12:09:49.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-17T12:09:49.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-17T12:09:49.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-17T12:09:49.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-17T12:09:49.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-17T12:09:49.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-17T12:09:49.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:49.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:49.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-17T12:09:49.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-17T12:09:50.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-17T12:09:50.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-17T12:09:50.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-17T12:09:50.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-17T12:09:50.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.096 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-17T12:09:50.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-17T12:09:50.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-17T12:09:50.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-17T12:09:50.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-17T12:09:50.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-17T12:09:50.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-17T12:09:50.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-17T12:09:50.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-17T12:09:50.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-17T12:09:50.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-17T12:09:50.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-17T12:09:50.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-17T12:09:50.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-17T12:09:50.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-17T12:09:50.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-17T12:09:50.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-17T12:09:50.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-17T12:09:50.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-17T12:09:50.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-17T12:09:50.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-17T12:09:50.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-17T12:09:50.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-17T12:09:50.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-17T12:09:50.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-17T12:09:50.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-17T12:09:50.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-17T12:09:50.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-17T12:09:50.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-17T12:09:50.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-17T12:09:50.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-17T12:09:50.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-17T12:09:50.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-17T12:09:50.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-17T12:09:50.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-17T12:09:50.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-17T12:09:50.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-17T12:09:50.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-17T12:09:50.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-17T12:09:50.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-17T12:09:50.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-17T12:09:50.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-17T12:09:50.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-17T12:09:50.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-17T12:09:50.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.908 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-17T12:09:50.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-17T12:09:50.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-17T12:09:50.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-17T12:09:50.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-17T12:09:50.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-17T12:09:50.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:50.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:50.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-17T12:09:50.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-17T12:09:51.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-17T12:09:51.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-17T12:09:51.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-17T12:09:51.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-17T12:09:51.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-17T12:09:51.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-17T12:09:51.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-17T12:09:51.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-17T12:09:51.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-17T12:09:51.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-17T12:09:51.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-17T12:09:51.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-17T12:09:51.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-17T12:09:51.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-17T12:09:51.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-17T12:09:51.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-17T12:09:51.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-17T12:09:51.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-17T12:09:51.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-17T12:09:51.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-17T12:09:51.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-17T12:09:51.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-17T12:09:51.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-17T12:09:51.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-17T12:09:51.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-17T12:09:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-17T12:09:51.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-17T12:09:51.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-17T12:09:51.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-17T12:09:51.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-17T12:09:51.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-17T12:09:51.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-17T12:09:51.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-17T12:09:51.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-17T12:09:51.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-17T12:09:51.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-17T12:09:51.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-17T12:09:51.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-17T12:09:51.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-17T12:09:51.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-17T12:09:51.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-17T12:09:51.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-17T12:09:51.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-17T12:09:51.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:591: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:09:51.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-17T12:09:51.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:597: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-17T12:09:51.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:602: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:09:51.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:09:51.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:09:51.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:09:52.071 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:09:52.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:09:52.278 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:52.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:52.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:52.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:52.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:52.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:09:52.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:09:52.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:09:52.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:09:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:09:52.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:09:52.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:52.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:09:52.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:09:52.401 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:52.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:09:52.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:09:52.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:09:52.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:09:52.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.442 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:09:52.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:52.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:52.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:52.446 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:09:52.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:09:52.446 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:09:52.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:09:52.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:09:52.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:09:52.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:52.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:09:52.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:09:52.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:52.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:09:52.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:09:52.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:09:52.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:09:52.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:09:52.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:09:52.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:09:52.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:09:52.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:09:52.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:09:52.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:09:52.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:09:52.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:09:52.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:52.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:09:52.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:09:52.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:09:52.504 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:09:52.550 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:09:52.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:09:52.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:09:52.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:09:52.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:09:52.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:09:52.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:09:52.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:09:52.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:09:52.695 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:09:52.695 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:52.695 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:52.696 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:52.696 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:52.696 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:52.696 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:52.696 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:09:52.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:09:52.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:09:52.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:09:52.808 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:09:52.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:53.816 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:53.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:09:53.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:09:53.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:09:53.857 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:09:53.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:09:53.863 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:09:52.493+0000 7f0d7e9cbd80 0 load: jerasure load: lrc 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:359: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:362: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:09:53.864 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-17T12:09:53.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:53.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:53.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2e4761b7-ae0a-419d-a93f-cdf414b8ed1e 2026-02-17T12:09:53.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2e4761b7-ae0a-419d-a93f-cdf414b8ed1e' 2026-02-17T12:09:53.870 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 2e4761b7-ae0a-419d-a93f-cdf414b8ed1e 2026-02-17T12:09:53.870 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:09:53.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCRWpRpxXF7NBAAk9ehvByaECkwVy69SMbhnw== 2026-02-17T12:09:53.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCRWpRpxXF7NBAAk9ehvByaECkwVy69SMbhnw=="}' 2026-02-17T12:09:53.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2e4761b7-ae0a-419d-a93f-cdf414b8ed1e -i td/test-erasure-eio/0/new.json 2026-02-17T12:09:53.985 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:09:53.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:09:53.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCRWpRpxXF7NBAAk9ehvByaECkwVy69SMbhnw== --osd-uuid 2e4761b7-ae0a-419d-a93f-cdf414b8ed1e 2026-02-17T12:09:54.010 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:54.009+0000 7f0e562d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:54.011 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:54.010+0000 7f0e562d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:54.012 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:54.011+0000 7f0e562d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:54.013 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:54.012+0000 7f0e562d9780 -1 bdev(0x55c3be3d3c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:09:54.013 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:54.012+0000 7f0e562d9780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:09:56.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:09:56.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:09:56.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:09:56.139 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:09:56.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:09:56.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:09:56.393 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:09:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:09:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:09:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:09:56.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:09:56.412 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:56.411+0000 7fca2ec14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:56.413 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:56.412+0000 7fca2ec14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:56.414 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:56.413+0000 7fca2ec14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:56.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:09:56.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:56.583 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:09:56.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:56.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:56.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:57.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:57.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:57.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:57.773 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:09:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:57.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:58.009 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:09:58.007+0000 7fca2ec14780 -1 Falling back to public interface 2026-02-17T12:09:58.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:58.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:58.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:58.963 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:09:58.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:58.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:59.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:00.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:00.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:00.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:00.154 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:00.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:00.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:00.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:00.710 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:00.708+0000 7fca2ec14780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:10:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:01.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:01.347 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:01.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:01.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:01.542 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1575674114,v1:127.0.0.1:6803/1575674114] [v2:127.0.0.1:6804/1575674114,v1:127.0.0.1:6805/1575674114] exists,up 2e4761b7-ae0a-419d-a93f-cdf414b8ed1e 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:10:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:10:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bbc1993e-1822-4543-8f94-e8a7dc6be2b9 2026-02-17T12:10:01.550 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 bbc1993e-1822-4543-8f94-e8a7dc6be2b9 2026-02-17T12:10:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 bbc1993e-1822-4543-8f94-e8a7dc6be2b9' 2026-02-17T12:10:01.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:01.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCZWpRpNP8jIRAAUcKeSYP6p60VTSPSQ5Be9A== 2026-02-17T12:10:01.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCZWpRpNP8jIRAAUcKeSYP6p60VTSPSQ5Be9A=="}' 2026-02-17T12:10:01.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bbc1993e-1822-4543-8f94-e8a7dc6be2b9 -i td/test-erasure-eio/1/new.json 2026-02-17T12:10:01.748 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:01.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:10:01.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZWpRpNP8jIRAAUcKeSYP6p60VTSPSQ5Be9A== --osd-uuid bbc1993e-1822-4543-8f94-e8a7dc6be2b9 2026-02-17T12:10:01.775 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:01.773+0000 7f3c789a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:01.776 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:01.774+0000 7f3c789a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:01.777 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:01.775+0000 7f3c789a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:01.778 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:01.775+0000 7f3c789a3780 -1 bdev(0x565060f1bc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:01.778 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:01.775+0000 7f3c789a3780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:10:03.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:10:03.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:03.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:10:03.907 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:10:03.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:04.171 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:10:04.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:10:04.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:04.188 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:04.186+0000 7f640e60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:04.190 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:04.187+0000 7f640e60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:04.190 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:04.188+0000 7f640e60c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:04.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:04.361 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:04.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:04.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:04.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:04.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:05.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:05.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:05.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:05.553 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:05.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:05.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:05.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:05.784 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:05.781+0000 7f640e60c780 -1 Falling back to public interface 2026-02-17T12:10:06.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:06.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:06.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:06.748 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:06.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:06.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:06.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:07.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:07.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:07.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:07.937 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:07.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:07.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:08.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:08.724 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:08.723+0000 7f640e60c780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:10:09.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:09.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:09.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:09.126 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:09.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:09.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:09.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:10.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:10.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:10.316 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:10:10.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:10:10.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:10.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:10.507 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/151049615,v1:127.0.0.1:6811/151049615] [v2:127.0.0.1:6812/151049615,v1:127.0.0.1:6813/151049615] exists,up bbc1993e-1822-4543-8f94-e8a7dc6be2b9 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:10.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:10.510 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:10.510 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:10.510 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:10.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:10.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:10:10.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:10.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=887cec90-2d58-466c-8e36-60abfbfe9cf8 2026-02-17T12:10:10.512 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 887cec90-2d58-466c-8e36-60abfbfe9cf8 2026-02-17T12:10:10.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 887cec90-2d58-466c-8e36-60abfbfe9cf8' 2026-02-17T12:10:10.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:10.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCiWpRpqYMSHxAAh8vFNmyxEwClEphlTLkAEA== 2026-02-17T12:10:10.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCiWpRpqYMSHxAAh8vFNmyxEwClEphlTLkAEA=="}' 2026-02-17T12:10:10.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 887cec90-2d58-466c-8e36-60abfbfe9cf8 -i td/test-erasure-eio/2/new.json 2026-02-17T12:10:10.719 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:10.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:10:10.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCiWpRpqYMSHxAAh8vFNmyxEwClEphlTLkAEA== --osd-uuid 887cec90-2d58-466c-8e36-60abfbfe9cf8 2026-02-17T12:10:10.746 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:10.744+0000 7f97c9a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:10.747 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:10.746+0000 7f97c9a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:10.748 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:10.747+0000 7f97c9a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:10.749 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:10.747+0000 7f97c9a2d780 -1 bdev(0x55eaec905c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:10.749 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:10.747+0000 7f97c9a2d780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:10:12.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:10:12.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:12.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:10:12.872 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:10:12.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:13.130 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:10:13.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:10:13.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:13.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:13.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:13.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:13.146 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:13.144+0000 7f4cb5aaf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:13.147 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:13.146+0000 7f4cb5aaf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:13.148 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:13.147+0000 7f4cb5aaf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:13.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:13.318 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:13.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:13.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:10:13.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:14.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:14.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:14.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:14.509 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:14.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:14.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:10:14.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:14.735 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:14.733+0000 7f4cb5aaf780 -1 Falling back to public interface 2026-02-17T12:10:15.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:15.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:15.699 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:15.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:15.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:15.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:10:15.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:16.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:16.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:16.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:16.890 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:16.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:16.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:10:17.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:17.675 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:17.674+0000 7f4cb5aaf780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:10:18.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:18.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:18.081 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:18.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:18.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:18.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:10:18.268 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2264190890,v1:127.0.0.1:6819/2264190890] [v2:127.0.0.1:6820/2264190890,v1:127.0.0.1:6821/2264190890] exists,up 887cec90-2d58-466c-8e36-60abfbfe9cf8 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:18.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:18.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:10:18.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5a01d46f-cdc8-4144-8875-abb79967def9 2026-02-17T12:10:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 5a01d46f-cdc8-4144-8875-abb79967def9' 2026-02-17T12:10:18.272 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 5a01d46f-cdc8-4144-8875-abb79967def9 2026-02-17T12:10:18.272 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:18.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCqWpRpuzrqEBAAr7+5eWyDgV+9JJKK/9+Vww== 2026-02-17T12:10:18.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCqWpRpuzrqEBAAr7+5eWyDgV+9JJKK/9+Vww=="}' 2026-02-17T12:10:18.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5a01d46f-cdc8-4144-8875-abb79967def9 -i td/test-erasure-eio/3/new.json 2026-02-17T12:10:18.479 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:18.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:10:18.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCqWpRpuzrqEBAAr7+5eWyDgV+9JJKK/9+Vww== --osd-uuid 5a01d46f-cdc8-4144-8875-abb79967def9 2026-02-17T12:10:18.506 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:18.504+0000 7f3b56421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:18.507 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:18.506+0000 7f3b56421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:18.508 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:18.507+0000 7f3b56421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:18.508 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:18.507+0000 7f3b56421780 -1 bdev(0x56306fd6fc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:18.508 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:18.507+0000 7f3b56421780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:10:20.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:10:20.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:20.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:10:20.622 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:10:20.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:20.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:10:20.894 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:10:20.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:20.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:20.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:20.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:20.911 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:20.910+0000 7fc4f2b51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:20.912 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:20.911+0000 7fc4f2b51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:20.913 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:20.912+0000 7fc4f2b51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:21.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:10:21.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:21.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:10:21.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:21.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:21.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:21.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:21.085 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:21.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:21.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:21.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:22.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:22.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:22.273 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:22.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:22.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:22.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:22.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:22.505 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:22.503+0000 7fc4f2b51780 -1 Falling back to public interface 2026-02-17T12:10:23.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:23.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:23.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:23.461 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:23.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:23.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:23.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:24.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:24.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:25.203 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:25.202+0000 7fc4f2b51780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:25.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:26.030 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1761539991,v1:127.0.0.1:6827/1761539991] [v2:127.0.0.1:6828/1761539991,v1:127.0.0.1:6829/1761539991] exists,up 5a01d46f-cdc8-4144-8875-abb79967def9 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:26.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:26.036 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:26.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8947eeae-b827-4b18-a40d-06bde6acbe8d 2026-02-17T12:10:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 8947eeae-b827-4b18-a40d-06bde6acbe8d' 2026-02-17T12:10:26.039 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 8947eeae-b827-4b18-a40d-06bde6acbe8d 2026-02-17T12:10:26.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:26.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCyWpRpQT2vAhAAI8bMHIwjllBxUIddYFqctw== 2026-02-17T12:10:26.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCyWpRpQT2vAhAAI8bMHIwjllBxUIddYFqctw=="}' 2026-02-17T12:10:26.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8947eeae-b827-4b18-a40d-06bde6acbe8d -i td/test-erasure-eio/4/new.json 2026-02-17T12:10:26.241 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:26.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:10:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCyWpRpQT2vAhAAI8bMHIwjllBxUIddYFqctw== --osd-uuid 8947eeae-b827-4b18-a40d-06bde6acbe8d 2026-02-17T12:10:26.268 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:26.266+0000 7f65ce833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:26.269 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:26.268+0000 7f65ce833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:26.270 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:26.269+0000 7f65ce833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:26.270 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:26.269+0000 7f65ce833780 -1 bdev(0x55be0d713c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:26.270 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:26.269+0000 7f65ce833780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:10:28.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:10:28.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:28.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:10:28.385 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:10:28.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:28.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:10:28.638 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:10:28.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:28.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:28.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:28.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:28.654 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:28.653+0000 7fe9a1e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:28.656 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:28.654+0000 7fe9a1e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:28.656 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:28.655+0000 7fe9a1e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:28.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:28.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:28.829 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:28.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:28.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:29.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:30.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:30.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:30.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:30.015 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:30.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:30.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:30.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:30.246 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:30.244+0000 7fe9a1e2c780 -1 Falling back to public interface 2026-02-17T12:10:31.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:31.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:31.207 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:31.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:31.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:31.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:31.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:32.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:32.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:32.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:32.401 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:32.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:32.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:32.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:33.196 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:33.195+0000 7fe9a1e2c780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:10:33.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:33.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:33.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:33.592 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:33.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:33.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:33.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:34.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:34.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:34.791 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:10:34.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:10:34.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:34.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:10:34.983 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/445984706,v1:127.0.0.1:6835/445984706] [v2:127.0.0.1:6836/445984706,v1:127.0.0.1:6837/445984706] exists,up 8947eeae-b827-4b18-a40d-06bde6acbe8d 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:34.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:34.988 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:34.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7b2523bd-a8cb-42f3-8cc5-ac4deac9676c 2026-02-17T12:10:34.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 7b2523bd-a8cb-42f3-8cc5-ac4deac9676c' 2026-02-17T12:10:34.991 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 7b2523bd-a8cb-42f3-8cc5-ac4deac9676c 2026-02-17T12:10:34.991 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:34.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC6WpRp0tdwOxAAuXYWbqNccV25qPfla88gzA== 2026-02-17T12:10:34.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC6WpRp0tdwOxAAuXYWbqNccV25qPfla88gzA=="}' 2026-02-17T12:10:34.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7b2523bd-a8cb-42f3-8cc5-ac4deac9676c -i td/test-erasure-eio/5/new.json 2026-02-17T12:10:35.190 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:10:35.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-17T12:10:35.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC6WpRp0tdwOxAAuXYWbqNccV25qPfla88gzA== --osd-uuid 7b2523bd-a8cb-42f3-8cc5-ac4deac9676c 2026-02-17T12:10:35.216 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:35.215+0000 7f57863d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:35.218 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:35.217+0000 7f57863d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:35.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:35.217+0000 7f57863d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:35.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:35.218+0000 7f57863d4780 -1 bdev(0x55634b433c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:35.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:35.218+0000 7f57863d4780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-17T12:10:37.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-17T12:10:37.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:37.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:10:37.829 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:10:37.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:38.084 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:10:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:10:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:38.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:38.101 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:38.100+0000 7f4a3dfb4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:38.102 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:38.101+0000 7f4a3dfb4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:38.103 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:38.102+0000 7f4a3dfb4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:38.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:10:38.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:38.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:10:38.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:38.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:38.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:38.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:38.281 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:38.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:38.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:10:38.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:39.198 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:39.196+0000 7f4a3dfb4780 -1 Falling back to public interface 2026-02-17T12:10:39.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:39.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:39.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:39.470 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:39.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:39.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:10:39.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:40.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:40.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:40.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:40.659 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:40.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:40.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:10:40.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:41.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:10:41.913 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:41.911+0000 7f4a3dfb4780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:10:42.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:43.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:43.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:43.048 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:43.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:43.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:43.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:10:43.239 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/757987206,v1:127.0.0.1:6843/757987206] [v2:127.0.0.1:6844/757987206,v1:127.0.0.1:6845/757987206] exists,up 7b2523bd-a8cb-42f3-8cc5-ac4deac9676c 2026-02-17T12:10:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:43.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:43.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5d4f0a3b-551e-4638-a56e-22bef8135435 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 5d4f0a3b-551e-4638-a56e-22bef8135435' 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 5d4f0a3b-551e-4638-a56e-22bef8135435 2026-02-17T12:10:43.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:43.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDDWpRpMQ8dDxAAwWa85htLzCuaoY23MuZ8+A== 2026-02-17T12:10:43.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDDWpRpMQ8dDxAAwWa85htLzCuaoY23MuZ8+A=="}' 2026-02-17T12:10:43.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5d4f0a3b-551e-4638-a56e-22bef8135435 -i td/test-erasure-eio/6/new.json 2026-02-17T12:10:43.446 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:10:43.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-17T12:10:43.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDDWpRpMQ8dDxAAwWa85htLzCuaoY23MuZ8+A== --osd-uuid 5d4f0a3b-551e-4638-a56e-22bef8135435 2026-02-17T12:10:43.472 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:43.470+0000 7f143052e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:43.474 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:43.472+0000 7f143052e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:43.475 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:43.473+0000 7f143052e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:43.475 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:43.473+0000 7f143052e780 -1 bdev(0x5606b7487c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:43.475 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:43.473+0000 7f143052e780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-17T12:10:45.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-17T12:10:45.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:45.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:10:45.599 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:10:45.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:45.865 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:10:45.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:10:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:45.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:45.883 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:45.880+0000 7f61343b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:45.884 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:45.882+0000 7f61343b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:45.885 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:45.883+0000 7f61343b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:46.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:10:46.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:46.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:10:46.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:46.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:46.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:46.057 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:46.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:46.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:46.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:10:46.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:47.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:47.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:47.254 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:10:47.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:47.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:47.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:10:47.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:47.467 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:47.465+0000 7f61343b4780 -1 Falling back to public interface 2026-02-17T12:10:48.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:48.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:48.444 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:10:48.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:48.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:48.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:10:48.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:49.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:49.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:49.633 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:10:49.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:49.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:49.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:10:49.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:50.163 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:50.162+0000 7f61343b4780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:10:50.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:50.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:50.816 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:10:50.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:50.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:50.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:10:51.008 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/885840473,v1:127.0.0.1:6851/885840473] [v2:127.0.0.1:6852/885840473,v1:127.0.0.1:6853/885840473] exists,up 5d4f0a3b-551e-4638-a56e-22bef8135435 2026-02-17T12:10:51.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:51.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:10:51.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:10:51.055 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:10:51.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:10:51.061 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:09:58.010+0000 7fca2ec14780 0 load: jerasure load: lrc 2026-02-17T12:10:51.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-17T12:10:51.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:10:51.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:10:51.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:10:51.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:10:51.628 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:10:51.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:10:52.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:10:52.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:10:52.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:10:52.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:10:52.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:10:52.640 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:10:52.640 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:10:52.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:10:52.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:10:52.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:10:52.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:52.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:10:52.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:10:52.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:10:52.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:10:52.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:52.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:10:53.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574859 2026-02-17T12:10:53.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574859 2026-02-17T12:10:53.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859' 2026-02-17T12:10:53.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:53.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:10:53.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378633 2026-02-17T12:10:53.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378633 2026-02-17T12:10:53.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859 2-81604378633' 2026-02-17T12:10:53.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:53.101 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:10:53.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117000 2026-02-17T12:10:53.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117000 2026-02-17T12:10:53.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859 2-81604378633 3-115964117000' 2026-02-17T12:10:53.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:53.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:10:53.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855366 2026-02-17T12:10:53.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855366 2026-02-17T12:10:53.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859 2-81604378633 3-115964117000 4-150323855366' 2026-02-17T12:10:53.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:53.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:10:53.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593733 2026-02-17T12:10:53.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593733 2026-02-17T12:10:53.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859 2-81604378633 3-115964117000 4-150323855366 5-184683593733' 2026-02-17T12:10:53.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:53.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:10:53.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332099 2026-02-17T12:10:53.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332099 2026-02-17T12:10:53.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-55834574859 2-81604378633 3-115964117000 4-150323855366 5-184683593733 6-219043332099' 2026-02-17T12:10:53.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:53.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:10:53.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:53.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:10:53.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:10:53.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:53.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:10:53.355 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803789 2026-02-17T12:10:53.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:10:53.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:53.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:10:53.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:10:54.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:10:54.555 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:54.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:10:54.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:10:55.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:10:55.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:55.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:10:55.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:55.931 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-55834574859 2026-02-17T12:10:55.931 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:55.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:10:55.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-55834574859 2026-02-17T12:10:55.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:55.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574859 2026-02-17T12:10:55.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 55834574859' 2026-02-17T12:10:55.934 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 55834574859 2026-02-17T12:10:55.934 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:10:56.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574859 -lt 55834574859 2026-02-17T12:10:56.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:56.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-81604378633 2026-02-17T12:10:56.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:56.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:10:56.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-81604378633 2026-02-17T12:10:56.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:56.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378633 2026-02-17T12:10:56.127 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 81604378633 2026-02-17T12:10:56.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 81604378633' 2026-02-17T12:10:56.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378634 -lt 81604378633 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-115964117000 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:56.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:10:56.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-115964117000 2026-02-17T12:10:56.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:56.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117000 2026-02-17T12:10:56.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 115964117000' 2026-02-17T12:10:56.318 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 115964117000 2026-02-17T12:10:56.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:10:56.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117000 -lt 115964117000 2026-02-17T12:10:56.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:56.509 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855366 2026-02-17T12:10:56.509 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:56.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:10:56.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855366 2026-02-17T12:10:56.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:56.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855366 2026-02-17T12:10:56.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855366' 2026-02-17T12:10:56.512 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855366 2026-02-17T12:10:56.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:10:56.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855367 -lt 150323855366 2026-02-17T12:10:56.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:56.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-184683593733 2026-02-17T12:10:56.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:56.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:10:56.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-184683593733 2026-02-17T12:10:56.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:56.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593733 2026-02-17T12:10:56.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 184683593733' 2026-02-17T12:10:56.705 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 184683593733 2026-02-17T12:10:56.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:10:56.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-17T12:10:56.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:56.895 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-219043332099 2026-02-17T12:10:56.895 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:56.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:10:56.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-219043332099 2026-02-17T12:10:56.896 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:56.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332099 2026-02-17T12:10:56.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-17T12:10:56.898 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 219043332099 2026-02-17T12:10:56.898 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:10:57.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332100 -lt 219043332099 2026-02-17T12:10:57.087 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:10:57.088 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:57.088 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:57.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:10:57.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:10:57.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:10:57.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:10:57.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:10:57.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:10:57.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:10:57.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:10:57.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:10:57.525 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:10:57.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:57.526 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:57.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:10:57.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:10:57.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:10:57.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-17T12:10:57.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=pool-jerasure 2026-02-17T12:10:57.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=myobject 2026-02-17T12:10:57.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:10:57.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:10:57.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:10:57.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:10:59.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-17T12:10:59.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:375: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:10:59.304 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:10:59.498 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2147483647 2026-02-17T12:10:59.498 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:10:59.498 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:10:59.498 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2147483647 5 1 0 6 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=5 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:10:59.499 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/5/type 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=5 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:10:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.5 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.5.asok 2026-02-17T12:10:59.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:10:59.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:10:59.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:10:59.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.5 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.5 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.5 ']' 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.5.asok 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:10:59.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-17T12:10:59.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:378: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-17T12:10:59.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:10:59.602 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:10:59.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:10:59.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:10:59.604 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:10:59.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:10:59.618 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:59.617+0000 7fc7c963d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:59.619 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:59.618+0000 7fc7c963d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:59.620 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:10:59.619+0000 7fc7c963d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:59.794 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:10:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:10:59.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:00.955 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:00.954+0000 7fc7c963d780 -1 Falling back to public interface 2026-02-17T12:11:00.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:00.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:00.983 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:00.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:00.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:00.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:01.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:02.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:02.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:02.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:02.179 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:02.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:02.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:02.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:03.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:03.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:03.371 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:03.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:03.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:03.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:03.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:03.670 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:03.669+0000 7fc7c963d780 -1 osd.3 58 log_to_monitors true 2026-02-17T12:11:04.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:04.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:04.559 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:04.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:04.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:04.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:04.744 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 62 up_thru 62 down_at 59 last_clean_interval [27,58) [v2:127.0.0.1:6826/2306639321,v1:127.0.0.1:6827/2306639321] [v2:127.0.0.1:6828/2306639321,v1:127.0.0.1:6829/2306639321] exists,up 5a01d46f-cdc8-4144-8875-abb79967def9 2026-02-17T12:11:04.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:04.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:04.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:381: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:11:04.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:11:04.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:11:04.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:11:04.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:11:04.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:11:04.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:11:04.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:11:05.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.017 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:11:05.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803793 2026-02-17T12:11:05.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803793 2026-02-17T12:11:05.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793' 2026-02-17T12:11:05.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:11:05.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T12:11:05.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T12:11:05.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864' 2026-02-17T12:11:05.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.140 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:11:05.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378638 2026-02-17T12:11:05.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378638 2026-02-17T12:11:05.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-81604378638' 2026-02-17T12:11:05.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:11:05.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=266287972355 2026-02-17T12:11:05.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 266287972355 2026-02-17T12:11:05.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-81604378638 3-266287972355' 2026-02-17T12:11:05.266 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.266 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:11:05.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855371 2026-02-17T12:11:05.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855371 2026-02-17T12:11:05.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-81604378638 3-266287972355 4-150323855371' 2026-02-17T12:11:05.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:11:05.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593737 2026-02-17T12:11:05.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593737 2026-02-17T12:11:05.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-81604378638 3-266287972355 4-150323855371 5-184683593737' 2026-02-17T12:11:05.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:05.390 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:11:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332103 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332103 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-81604378638 3-266287972355 4-150323855371 5-184683593737 6-219043332103' 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803793 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:05.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:11:05.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803793 2026-02-17T12:11:05.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:05.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803793 2026-02-17T12:11:05.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-17T12:11:05.455 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803793 2026-02-17T12:11:05.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:05.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-17T12:11:05.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:06.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:11:06.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:06.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-17T12:11:06.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:06.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-55834574864 2026-02-17T12:11:06.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:06.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:11:06.837 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-55834574864 2026-02-17T12:11:06.837 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:06.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T12:11:06.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 55834574864' 2026-02-17T12:11:06.839 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 55834574864 2026-02-17T12:11:06.839 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:11:07.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-17T12:11:07.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:07.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-81604378638 2026-02-17T12:11:07.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:07.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:11:07.026 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-81604378638 2026-02-17T12:11:07.026 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:07.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378638 2026-02-17T12:11:07.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 81604378638' 2026-02-17T12:11:07.028 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 81604378638 2026-02-17T12:11:07.028 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:11:07.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378638 -lt 81604378638 2026-02-17T12:11:07.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:07.215 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-266287972355 2026-02-17T12:11:07.215 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:07.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:11:07.216 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-266287972355 2026-02-17T12:11:07.216 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:07.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=266287972355 2026-02-17T12:11:07.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 266287972355' 2026-02-17T12:11:07.218 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 266287972355 2026-02-17T12:11:07.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:11:07.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 266287972355 -lt 266287972355 2026-02-17T12:11:07.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:07.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855371 2026-02-17T12:11:07.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:07.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:11:07.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855371 2026-02-17T12:11:07.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:07.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855371 2026-02-17T12:11:07.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855371' 2026-02-17T12:11:07.411 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855371 2026-02-17T12:11:07.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:11:07.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855371 -lt 150323855371 2026-02-17T12:11:07.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:07.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-184683593737 2026-02-17T12:11:07.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:07.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:11:07.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-184683593737 2026-02-17T12:11:07.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:07.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593737 2026-02-17T12:11:07.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 184683593737' 2026-02-17T12:11:07.602 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 184683593737 2026-02-17T12:11:07.602 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:11:07.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593737 -lt 184683593737 2026-02-17T12:11:07.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:07.789 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-219043332103 2026-02-17T12:11:07.790 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:07.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:11:07.791 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-219043332103 2026-02-17T12:11:07.791 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:07.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332103 2026-02-17T12:11:07.792 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 219043332103 2026-02-17T12:11:07.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 219043332103' 2026-02-17T12:11:07.792 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:11:07.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332104 -lt 219043332103 2026-02-17T12:11:07.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:11:07.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:07.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:08.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:11:08.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:11:08.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:11:08.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:11:08.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:08.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:383: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:11:08.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-17T12:11:08.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:11:08.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:11:08.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:11:08.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:11:08.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:11:08.886 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:11:08.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:11:09.096 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:11:09.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:11:09.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:11:09.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:11:09.216 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:11:09.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:11:09.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:11:09.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:11:09.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:11:09.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:11:09.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:09.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:11:09.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:11:09.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:09.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:11:09.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:11:09.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:11:09.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:11:09.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:11:09.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:11:09.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:11:09.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:11:09.243 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:11:09.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:11:09.243 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:11:09.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:11:09.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:11:09.245 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:11:09.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:09.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:11:09.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:11:09.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:09.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:11:09.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:11:09.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:11:09.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:11:09.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:11:09.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:11:09.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:11:09.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:11:09.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:11:09.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:11:09.252 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:11:09.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:11:09.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:11:09.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:11:09.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:11:09.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:11:09.273 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:11:09.273 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:09.273 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:09.273 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:09.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:09.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:11:09.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:11:09.302 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:11:09.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:11:09.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:11:09.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:11:09.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:11:09.347 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:11:09.347 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:11:09.348 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:11:09.348 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:11:09.348 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:11:09.348 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:11:09.348 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.349 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.349 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:11:09.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:11:09.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:11:09.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:11:09.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:09.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:09.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:11:09.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:11:09.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:11:09.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:11:09.606 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:11:09.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:11:10.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:11:10.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:11:10.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:11:10.615 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:11:10.616 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:10.616 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:10.616 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:11:10.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:11:10.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:11:10.656 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:11:10.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:11:10.662 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:11:09.291+0000 7f13e261ad80 0 load: jerasure load: lrc 2026-02-17T12:11:10.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:11:10.664 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:10.670 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:10.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=85bbaac0-c516-4ff0-844a-ad8a18533da3 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 85bbaac0-c516-4ff0-844a-ad8a18533da3' 2026-02-17T12:11:10.672 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 85bbaac0-c516-4ff0-844a-ad8a18533da3 2026-02-17T12:11:10.673 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:10.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDeWpRpczOIKBAA6iNF+oOtqomJ0MAPIqXPgQ== 2026-02-17T12:11:10.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDeWpRpczOIKBAA6iNF+oOtqomJ0MAPIqXPgQ=="}' 2026-02-17T12:11:10.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 85bbaac0-c516-4ff0-844a-ad8a18533da3 -i td/test-erasure-eio/0/new.json 2026-02-17T12:11:10.785 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:10.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:11:10.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeWpRpczOIKBAA6iNF+oOtqomJ0MAPIqXPgQ== --osd-uuid 85bbaac0-c516-4ff0-844a-ad8a18533da3 2026-02-17T12:11:10.811 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:10.810+0000 7f439b7f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:10.812 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:10.811+0000 7f439b7f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:10.813 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:10.812+0000 7f439b7f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:10.814 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:10.812+0000 7f439b7f9780 -1 bdev(0x55b88933fc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:10.814 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:10.812+0000 7f439b7f9780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:11:12.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:11:12.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:12.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:11:12.944 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:11:12.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:13.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:11:13.200 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:11:13.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:13.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:13.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:13.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:13.217 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:13.216+0000 7f303cf84780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:13.218 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:13.217+0000 7f303cf84780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:13.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:13.218+0000 7f303cf84780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:13.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:13.391 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:13.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:13.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:13.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:13.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:14.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:14.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:14.579 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:14.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:14.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:14.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:14.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:14.827 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:14.826+0000 7f303cf84780 -1 Falling back to public interface 2026-02-17T12:11:15.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:15.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:15.770 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:15.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:15.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:15.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:15.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:16.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:16.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:16.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:16.959 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:16.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:16.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:17.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:17.765 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:17.764+0000 7f303cf84780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:18.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:18.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:19.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:19.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:19.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:11:19.347 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:11:19.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:19.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:19.537 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3315778933,v1:127.0.0.1:6803/3315778933] [v2:127.0.0.1:6804/3315778933,v1:127.0.0.1:6805/3315778933] exists,up 85bbaac0-c516-4ff0-844a-ad8a18533da3 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:19.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:19.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc' 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc 2026-02-17T12:11:19.541 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:19.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDnWpRp9rjuIBAAg+fxnzFwKLxPRueiNditGg== 2026-02-17T12:11:19.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDnWpRp9rjuIBAAg+fxnzFwKLxPRueiNditGg=="}' 2026-02-17T12:11:19.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc -i td/test-erasure-eio/1/new.json 2026-02-17T12:11:19.757 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:19.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:11:19.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDnWpRp9rjuIBAAg+fxnzFwKLxPRueiNditGg== --osd-uuid ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc 2026-02-17T12:11:19.782 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:19.780+0000 7f14bc411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:19.783 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:19.781+0000 7f14bc411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:19.784 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:19.782+0000 7f14bc411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:19.785 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:19.782+0000 7f14bc411780 -1 bdev(0x55b5c7403c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:19.785 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:19.782+0000 7f14bc411780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:11:21.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:11:21.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:21.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:11:21.901 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:11:21.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:22.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:11:22.158 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:11:22.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:22.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:22.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:22.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:22.174 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:22.172+0000 7ff0d1220780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:22.175 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:22.173+0000 7ff0d1220780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:22.176 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:22.174+0000 7ff0d1220780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:22.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:22.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:22.348 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:22.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:22.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:11:22.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:23.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:23.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:23.544 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:23.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:23.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:23.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:11:23.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:23.760 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:23.757+0000 7ff0d1220780 -1 Falling back to public interface 2026-02-17T12:11:24.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:24.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:24.733 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:24.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:24.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:24.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:11:24.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:25.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:25.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:25.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:25.928 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:25.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:25.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:11:26.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:26.464 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:26.461+0000 7ff0d1220780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:11:27.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:27.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:27.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:27.119 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:27.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:27.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:11:27.309 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/966065939,v1:127.0.0.1:6811/966065939] [v2:127.0.0.1:6812/966065939,v1:127.0.0.1:6813/966065939] exists,up ecebeed2-f2bb-4e85-8bf9-c69c1b0258dc 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:27.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:27.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1d4439f9-0f26-452b-a910-7289f32a7745 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 1d4439f9-0f26-452b-a910-7289f32a7745' 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 1d4439f9-0f26-452b-a910-7289f32a7745 2026-02-17T12:11:27.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDvWpRp5tJSExAAptbtyNevX7Sc99z54nplLA== 2026-02-17T12:11:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDvWpRp5tJSExAAptbtyNevX7Sc99z54nplLA=="}' 2026-02-17T12:11:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1d4439f9-0f26-452b-a910-7289f32a7745 -i td/test-erasure-eio/2/new.json 2026-02-17T12:11:27.517 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:27.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:11:27.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDvWpRp5tJSExAAptbtyNevX7Sc99z54nplLA== --osd-uuid 1d4439f9-0f26-452b-a910-7289f32a7745 2026-02-17T12:11:27.542 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:27.541+0000 7fa082421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:27.543 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:27.542+0000 7fa082421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:27.544 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:27.543+0000 7fa082421780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:27.545 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:27.543+0000 7fa082421780 -1 bdev(0x55c7eba01c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:27.545 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:27.543+0000 7fa082421780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:11:29.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:11:29.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:29.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:11:29.910 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:11:29.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:30.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:11:30.170 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:11:30.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:30.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:30.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:30.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:30.187 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:30.186+0000 7fe286323780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:30.189 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:30.187+0000 7fe286323780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:30.189 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:30.188+0000 7fe286323780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:30.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:30.366 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:30.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:30.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:30.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:11:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:31.520 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:31.518+0000 7fe286323780 -1 Falling back to public interface 2026-02-17T12:11:31.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:31.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:31.554 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:31.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:31.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:31.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:11:31.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:32.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:32.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:32.746 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:32.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:32.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:32.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:11:32.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:33.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:33.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:33.942 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:33.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:33.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:33.942 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:11:34.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:34.206 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:34.204+0000 7fe286323780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:11:35.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:35.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:35.132 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:35.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:35.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:35.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:11:35.322 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2467117744,v1:127.0.0.1:6819/2467117744] [v2:127.0.0.1:6820/2467117744,v1:127.0.0.1:6821/2467117744] exists,up 1d4439f9-0f26-452b-a910-7289f32a7745 2026-02-17T12:11:35.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:35.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:35.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:35.328 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:35.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ace22e52-e146-41f6-b2d8-b00c4b0bcc6f 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 ace22e52-e146-41f6-b2d8-b00c4b0bcc6f' 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 ace22e52-e146-41f6-b2d8-b00c4b0bcc6f 2026-02-17T12:11:35.330 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:35.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD3WpRpZ9UqFBAAtixMp3vXwItvbgxiY9Fgww== 2026-02-17T12:11:35.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD3WpRpZ9UqFBAAtixMp3vXwItvbgxiY9Fgww=="}' 2026-02-17T12:11:35.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ace22e52-e146-41f6-b2d8-b00c4b0bcc6f -i td/test-erasure-eio/3/new.json 2026-02-17T12:11:35.533 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:35.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:11:35.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3WpRpZ9UqFBAAtixMp3vXwItvbgxiY9Fgww== --osd-uuid ace22e52-e146-41f6-b2d8-b00c4b0bcc6f 2026-02-17T12:11:35.559 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:35.558+0000 7f214974f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:35.561 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:35.559+0000 7f214974f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:35.562 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:35.560+0000 7f214974f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:35.562 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:35.561+0000 7f214974f780 -1 bdev(0x562bd11f9c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:35.562 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:35.561+0000 7f214974f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:11:37.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:11:37.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:37.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:11:37.690 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:11:37.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:37.948 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:11:37.948 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:11:37.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:37.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:37.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:37.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:37.965 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:37.963+0000 7fda54319780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:37.966 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:37.965+0000 7fda54319780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:37.967 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:37.966+0000 7fda54319780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:38.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:38.137 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:38.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:38.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:38.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:39.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:39.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:39.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:39.328 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:39.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:39.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:39.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:39.557 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:39.555+0000 7fda54319780 -1 Falling back to public interface 2026-02-17T12:11:40.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:40.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:40.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:40.514 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:40.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:40.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:40.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:41.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:41.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:41.703 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:41.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:41.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:41.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:41.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:42.249 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:42.248+0000 7fda54319780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:11:42.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:42.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:42.890 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:42.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:42.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:42.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:11:43.084 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/59551273,v1:127.0.0.1:6827/59551273] [v2:127.0.0.1:6828/59551273,v1:127.0.0.1:6829/59551273] exists,up ace22e52-e146-41f6-b2d8-b00c4b0bcc6f 2026-02-17T12:11:43.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:43.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:43.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:43.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:43.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:11:43.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:43.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:43.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:43.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:43.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:43.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:11:43.095 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:43.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2097778a-7726-4fc3-bfca-3b12a3914fc7 2026-02-17T12:11:43.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 2097778a-7726-4fc3-bfca-3b12a3914fc7' 2026-02-17T12:11:43.095 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 2097778a-7726-4fc3-bfca-3b12a3914fc7 2026-02-17T12:11:43.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:43.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD/WpRpfPrgBRAAxbYQV+0p5p80ohxKuR17Sw== 2026-02-17T12:11:43.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD/WpRpfPrgBRAAxbYQV+0p5p80ohxKuR17Sw=="}' 2026-02-17T12:11:43.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2097778a-7726-4fc3-bfca-3b12a3914fc7 -i td/test-erasure-eio/4/new.json 2026-02-17T12:11:43.291 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:43.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:11:43.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD/WpRpfPrgBRAAxbYQV+0p5p80ohxKuR17Sw== --osd-uuid 2097778a-7726-4fc3-bfca-3b12a3914fc7 2026-02-17T12:11:43.318 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:43.317+0000 7fb5366d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:43.319 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:43.318+0000 7fb5366d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:43.320 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:43.319+0000 7fb5366d9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:43.321 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:43.319+0000 7fb5366d9780 -1 bdev(0x560848517c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:43.321 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:43.319+0000 7fb5366d9780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:11:45.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:11:45.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:45.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:11:45.440 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:11:45.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:45.696 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:11:45.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:11:45.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:45.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:45.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:45.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:45.713 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:45.712+0000 7f322e537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:45.715 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:45.713+0000 7f322e537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:45.716 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:45.714+0000 7f322e537780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:45.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:11:46.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:47.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:47.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:47.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:47.077 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:47.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:47.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:11:47.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:47.302 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:47.300+0000 7f322e537780 -1 Falling back to public interface 2026-02-17T12:11:48.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:48.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:48.274 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:11:48.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:49.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:49.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:49.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:49.463 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:49.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:49.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:11:49.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:50.011 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:50.009+0000 7f322e537780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:11:50.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:50.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:50.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:50.652 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:50.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:50.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:11:50.843 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1981106139,v1:127.0.0.1:6835/1981106139] [v2:127.0.0.1:6836/1981106139,v1:127.0.0.1:6837/1981106139] exists,up 2097778a-7726-4fc3-bfca-3b12a3914fc7 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:50.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:50.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:50.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=23d80585-44ce-4d82-a9b0-b77bf24e1fd9 2026-02-17T12:11:50.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 23d80585-44ce-4d82-a9b0-b77bf24e1fd9' 2026-02-17T12:11:50.851 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 23d80585-44ce-4d82-a9b0-b77bf24e1fd9 2026-02-17T12:11:50.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:50.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAGW5Rp7P0jMxAAqgFVPPGISXmLJHesjAXdnA== 2026-02-17T12:11:50.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAGW5Rp7P0jMxAAqgFVPPGISXmLJHesjAXdnA=="}' 2026-02-17T12:11:50.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 23d80585-44ce-4d82-a9b0-b77bf24e1fd9 -i td/test-erasure-eio/5/new.json 2026-02-17T12:11:51.053 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:11:51.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-17T12:11:51.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAGW5Rp7P0jMxAAqgFVPPGISXmLJHesjAXdnA== --osd-uuid 23d80585-44ce-4d82-a9b0-b77bf24e1fd9 2026-02-17T12:11:51.078 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:51.077+0000 7f1cf6e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:51.080 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:51.079+0000 7f1cf6e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:51.081 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:51.080+0000 7f1cf6e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:51.081 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:51.080+0000 7f1cf6e0c780 -1 bdev(0x55cba7b43c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:51.081 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:51.080+0000 7f1cf6e0c780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-17T12:11:53.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-17T12:11:53.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:53.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:11:53.204 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:11:53.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:53.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:11:53.458 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:11:53.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:11:53.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:53.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:53.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:53.475 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:53.474+0000 7f5cbc9f5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:53.476 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:53.475+0000 7f5cbc9f5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:53.477 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:53.476+0000 7f5cbc9f5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:53.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:11:53.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:53.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:53.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:54.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:54.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:54.837 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:11:54.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:54.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:54.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:55.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:55.080 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:55.079+0000 7f5cbc9f5780 -1 Falling back to public interface 2026-02-17T12:11:56.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:56.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:56.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:56.027 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:11:56.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:56.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:56.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:57.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:57.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:57.217 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:11:57.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:57.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:57.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:57.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:58.037 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:11:58.036+0000 7f5cbc9f5780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:11:58.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:58.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:58.408 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:11:58.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:58.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:58.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:58.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:59.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:59.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:59.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:11:59.600 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:11:59.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:59.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:11:59.789 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/27962137,v1:127.0.0.1:6843/27962137] [v2:127.0.0.1:6844/27962137,v1:127.0.0.1:6845/27962137] exists,up 23d80585-44ce-4d82-a9b0-b77bf24e1fd9 2026-02-17T12:11:59.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-17T12:11:59.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:59.795 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:59.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5833ebe2-8c95-4f59-bfa3-2f88a2206b32 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 5833ebe2-8c95-4f59-bfa3-2f88a2206b32' 2026-02-17T12:11:59.797 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 5833ebe2-8c95-4f59-bfa3-2f88a2206b32 2026-02-17T12:11:59.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:59.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAPW5RpGCjtLxAAFxE/6tCzewv2HiZb0cQCoQ== 2026-02-17T12:11:59.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAPW5RpGCjtLxAAFxE/6tCzewv2HiZb0cQCoQ=="}' 2026-02-17T12:11:59.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5833ebe2-8c95-4f59-bfa3-2f88a2206b32 -i td/test-erasure-eio/6/new.json 2026-02-17T12:11:59.998 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:12:00.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-17T12:12:00.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAPW5RpGCjtLxAAFxE/6tCzewv2HiZb0cQCoQ== --osd-uuid 5833ebe2-8c95-4f59-bfa3-2f88a2206b32 2026-02-17T12:12:00.022 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:00.020+0000 7fe0bd3d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:00.023 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:00.021+0000 7fe0bd3d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:00.024 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:00.022+0000 7fe0bd3d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:00.025 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:00.023+0000 7fe0bd3d4780 -1 bdev(0x562713305c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:00.025 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:00.023+0000 7fe0bd3d4780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-17T12:12:02.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-17T12:12:02.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:02.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:12:02.133 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:12:02.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:02.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:12:02.384 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:12:02.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:12:02.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:02.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:02.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:02.401 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:02.398+0000 7fcf4f40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:02.402 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:02.400+0000 7fcf4f40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:02.403 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:02.401+0000 7fcf4f40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:02.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:02.577 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:02.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:02.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:02.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:03.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:03.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:03.765 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:03.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:03.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:03.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:03.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:03.996 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:03.993+0000 7fcf4f40e780 -1 Falling back to public interface 2026-02-17T12:12:04.953 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:04.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:04.954 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:04.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:04.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:04.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:05.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:06.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:06.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:06.144 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:12:06.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:06.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:06.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:06.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:06.939 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:06.938+0000 7fcf4f40e780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:12:07.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:07.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:07.339 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:12:07.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:07.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:07.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:07.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:08.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:08.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:08.532 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:12:08.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:08.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:08.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:12:08.723 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2497355721,v1:127.0.0.1:6851/2497355721] [v2:127.0.0.1:6852/2497355721,v1:127.0.0.1:6853/2497355721] exists,up 5833ebe2-8c95-4f59-bfa3-2f88a2206b32 2026-02-17T12:12:08.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:08.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:08.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:08.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-17T12:12:08.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:08.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1deb9613-c58a-4fd7-8c9b-61b370513725 2026-02-17T12:12:08.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 1deb9613-c58a-4fd7-8c9b-61b370513725' 2026-02-17T12:12:08.731 INFO:tasks.workunit.client.0.trial044.stdout:add osd7 1deb9613-c58a-4fd7-8c9b-61b370513725 2026-02-17T12:12:08.731 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:08.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAYW5RpVub9KxAA8zct/KLf21nHCITAznRCBg== 2026-02-17T12:12:08.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAYW5RpVub9KxAA8zct/KLf21nHCITAznRCBg=="}' 2026-02-17T12:12:08.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1deb9613-c58a-4fd7-8c9b-61b370513725 -i td/test-erasure-eio/7/new.json 2026-02-17T12:12:08.932 INFO:tasks.workunit.client.0.trial044.stdout:7 2026-02-17T12:12:08.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-17T12:12:08.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAYW5RpVub9KxAA8zct/KLf21nHCITAznRCBg== --osd-uuid 1deb9613-c58a-4fd7-8c9b-61b370513725 2026-02-17T12:12:08.958 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:08.957+0000 7f20a17a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:08.959 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:08.958+0000 7f20a17a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:08.960 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:08.959+0000 7f20a17a3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:08.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:08.959+0000 7f20a17a3780 -1 bdev(0x562895015c00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:08.961 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:08.959+0000 7f20a17a3780 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-17T12:12:11.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-17T12:12:11.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:11.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T12:12:11.077 INFO:tasks.workunit.client.0.trial044.stdout:adding osd7 key to auth repository 2026-02-17T12:12:11.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:11.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T12:12:11.345 INFO:tasks.workunit.client.0.trial044.stdout:start osd.7 2026-02-17T12:12:11.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:12:11.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:11.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:11.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:11.363 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:11.362+0000 7fae8613f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:11.364 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:11.363+0000 7fae8613f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:11.365 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:11.364+0000 7fae8613f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:11.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:11.536 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:11.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:11.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:11.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:12.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:12.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:12.722 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:12.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:12.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:12.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:12.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:12.941 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:12.939+0000 7fae8613f780 -1 Falling back to public interface 2026-02-17T12:12:13.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:13.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:13.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:13.913 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:13.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:13.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:14.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:15.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:15.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:15.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:15.103 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:12:15.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:15.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:15.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:15.896 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:15.894+0000 7fae8613f780 -1 osd.7 0 log_to_monitors true 2026-02-17T12:12:16.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:16.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:16.299 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:12:16.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:16.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:16.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:16.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:17.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:17.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:17.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:17.487 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:12:17.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:17.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T12:12:17.677 INFO:tasks.workunit.client.0.trial044.stdout:osd.7 up in weight 1 up_from 55 up_thru 57 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/4069808776,v1:127.0.0.1:6859/4069808776] [v2:127.0.0.1:6860/4069808776,v1:127.0.0.1:6861/4069808776] exists,up 1deb9613-c58a-4fd7-8c9b-61b370513725 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:12:17.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:17.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:17.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=07c0dc29-5dc5-4482-a0da-76aeb4ffec8a 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 07c0dc29-5dc5-4482-a0da-76aeb4ffec8a' 2026-02-17T12:12:17.681 INFO:tasks.workunit.client.0.trial044.stdout:add osd8 07c0dc29-5dc5-4482-a0da-76aeb4ffec8a 2026-02-17T12:12:17.682 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:17.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAhW5Rp+sc7KRAAGgGn8T8oBs2qHmtAFsUYsQ== 2026-02-17T12:12:17.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAhW5Rp+sc7KRAAGgGn8T8oBs2qHmtAFsUYsQ=="}' 2026-02-17T12:12:17.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 07c0dc29-5dc5-4482-a0da-76aeb4ffec8a -i td/test-erasure-eio/8/new.json 2026-02-17T12:12:17.884 INFO:tasks.workunit.client.0.trial044.stdout:8 2026-02-17T12:12:17.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-17T12:12:17.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAhW5Rp+sc7KRAAGgGn8T8oBs2qHmtAFsUYsQ== --osd-uuid 07c0dc29-5dc5-4482-a0da-76aeb4ffec8a 2026-02-17T12:12:17.909 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:17.908+0000 7f76dba1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:17.910 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:17.909+0000 7f76dba1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:17.911 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:17.910+0000 7f76dba1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:17.912 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:17.911+0000 7f76dba1c780 -1 bdev(0x5625db3a5c00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:17.912 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:17.911+0000 7f76dba1c780 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-17T12:12:20.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-17T12:12:20.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:20.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T12:12:20.023 INFO:tasks.workunit.client.0.trial044.stdout:adding osd8 key to auth repository 2026-02-17T12:12:20.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:20.280 INFO:tasks.workunit.client.0.trial044.stdout:start osd.8 2026-02-17T12:12:20.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T12:12:20.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:12:20.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:20.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:20.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:20.297 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:20.296+0000 7f370162f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:20.298 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:20.297+0000 7f370162f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:20.300 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:20.298+0000 7f370162f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:20.468 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:20.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:20.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:20.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:21.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:21.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:21.665 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:21.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:21.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:21.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:21.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:21.899 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:21.898+0000 7f370162f780 -1 Falling back to public interface 2026-02-17T12:12:22.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:22.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:22.857 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:22.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:22.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:22.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:23.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:24.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:24.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:24.046 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:12:24.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:24.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:24.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:24.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:24.843 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:24.842+0000 7f370162f780 -1 osd.8 0 log_to_monitors true 2026-02-17T12:12:25.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:25.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:25.232 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:12:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:25.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:26.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:26.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:26.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:26.428 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:12:26.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:26.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T12:12:26.615 INFO:tasks.workunit.client.0.trial044.stdout:osd.8 up in weight 1 up_from 64 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2076008347,v1:127.0.0.1:6867/2076008347] [v2:127.0.0.1:6868/2076008347,v1:127.0.0.1:6869/2076008347] exists,up 07c0dc29-5dc5-4482-a0da-76aeb4ffec8a 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:26.617 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:26.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:12:26.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:12:26.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:12:26.664 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:12:26.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:12:26.672 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:11:14.829+0000 7f303cf84780 0 load: jerasure load: lrc 2026-02-17T12:12:26.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:423: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-17T12:12:26.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-17T12:12:26.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:12:26.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:12:26.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-17T12:12:26.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:12:26.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-17T12:12:26.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:12:26.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-17T12:12:26.922 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:12:26.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:12:27.215 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:12:27.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:28.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:12:28.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:28.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:28.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:28.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:28.227 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:28.228 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:28.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:28.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:28.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:28.312 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:28.500 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr:8' 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.501 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:28.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803792 2026-02-17T12:12:28.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803792 2026-02-17T12:12:28.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792' 2026-02-17T12:12:28.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:12:28.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574863 2026-02-17T12:12:28.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574863 2026-02-17T12:12:28.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863' 2026-02-17T12:12:28.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.626 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:12:28.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411342 2026-02-17T12:12:28.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411342 2026-02-17T12:12:28.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342' 2026-02-17T12:12:28.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:12:28.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149708 2026-02-17T12:12:28.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149708 2026-02-17T12:12:28.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708' 2026-02-17T12:12:28.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.752 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:12:28.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888074 2026-02-17T12:12:28.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888074 2026-02-17T12:12:28.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708 4-146028888074' 2026-02-17T12:12:28.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:12:28.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593737 2026-02-17T12:12:28.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593737 2026-02-17T12:12:28.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708 4-146028888074 5-184683593737' 2026-02-17T12:12:28.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:12:28.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397512 2026-02-17T12:12:28.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397512 2026-02-17T12:12:28.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708 4-146028888074 5-184683593737 6-210453397512' 2026-02-17T12:12:28.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:28.944 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:12:29.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201285 2026-02-17T12:12:29.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201285 2026-02-17T12:12:29.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708 4-146028888074 5-184683593737 6-210453397512 7-236223201285' 2026-02-17T12:12:29.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:29.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:12:29.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906947 2026-02-17T12:12:29.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906947 2026-02-17T12:12:29.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-77309411342 3-111669149708 4-146028888074 5-184683593737 6-210453397512 7-236223201285 8-274877906947' 2026-02-17T12:12:29.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:29.069 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803792 2026-02-17T12:12:29.069 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:29.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:29.070 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803792 2026-02-17T12:12:29.070 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:29.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803792 2026-02-17T12:12:29.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-17T12:12:29.071 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803792 2026-02-17T12:12:29.072 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:29.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-17T12:12:29.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:30.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:30.262 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:30.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803792 2026-02-17T12:12:30.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:30.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-55834574863 2026-02-17T12:12:30.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:30.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:12:30.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-55834574863 2026-02-17T12:12:30.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:30.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574863 2026-02-17T12:12:30.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 55834574863' 2026-02-17T12:12:30.453 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 55834574863 2026-02-17T12:12:30.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:12:30.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-17T12:12:30.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:30.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411342 2026-02-17T12:12:30.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:30.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:12:30.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411342 2026-02-17T12:12:30.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:30.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411342 2026-02-17T12:12:30.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-17T12:12:30.650 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411342 2026-02-17T12:12:30.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:12:30.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411342 -lt 77309411342 2026-02-17T12:12:30.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:30.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149708 2026-02-17T12:12:30.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:30.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:12:30.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149708 2026-02-17T12:12:30.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:30.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149708 2026-02-17T12:12:30.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149708' 2026-02-17T12:12:30.846 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149708 2026-02-17T12:12:30.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:12:31.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-17T12:12:31.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.036 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888074 2026-02-17T12:12:31.036 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:12:31.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888074 2026-02-17T12:12:31.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888074 2026-02-17T12:12:31.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888074' 2026-02-17T12:12:31.039 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888074 2026-02-17T12:12:31.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:12:31.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888074 -lt 146028888074 2026-02-17T12:12:31.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-184683593737 2026-02-17T12:12:31.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:12:31.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-184683593737 2026-02-17T12:12:31.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593737 2026-02-17T12:12:31.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 184683593737' 2026-02-17T12:12:31.232 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 184683593737 2026-02-17T12:12:31.232 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:12:31.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593738 -lt 184683593737 2026-02-17T12:12:31.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-210453397512 2026-02-17T12:12:31.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:12:31.424 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-210453397512 2026-02-17T12:12:31.425 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397512 2026-02-17T12:12:31.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 210453397512' 2026-02-17T12:12:31.426 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 210453397512 2026-02-17T12:12:31.426 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:12:31.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397512 -lt 210453397512 2026-02-17T12:12:31.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.616 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-236223201285 2026-02-17T12:12:31.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T12:12:31.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-236223201285 2026-02-17T12:12:31.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201285 2026-02-17T12:12:31.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 236223201285' 2026-02-17T12:12:31.619 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.7 seq 236223201285 2026-02-17T12:12:31.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T12:12:31.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201285 -lt 236223201285 2026-02-17T12:12:31.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.809 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-274877906947 2026-02-17T12:12:31.809 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:12:31.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-274877906947 2026-02-17T12:12:31.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906947 2026-02-17T12:12:31.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 274877906947' 2026-02-17T12:12:31.812 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 274877906947 2026-02-17T12:12:31.812 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:12:32.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906948 -lt 274877906947 2026-02-17T12:12:32.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:32.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:32.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:32.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:12:32.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:32.256 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:32.256 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:32.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:32.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:32.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:32.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:32.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:12:32.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:32.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:32.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:32.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:12:32.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:32.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:32.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:12:32.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-17T12:12:32.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:427: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:12:32.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:12:32.730 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:12:32.731 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:12:32.731 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:12:32.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:12:32.923 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:32.923 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:12:32.924 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:12:32.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:12:32.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:12:32.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:12:32.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:12:32.927 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:12:32.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:12:32.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:12:32.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:12:32.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:32.977 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:12:32.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:12:32.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:430: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:12:33.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=3 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:12:33.029 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:12:33.221 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:12:33.221 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:33.221 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:33.221 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:12:33.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/0/type 2026-02-17T12:12:33.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:12:33.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T12:12:33.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:12:33.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:12:33.224 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:33.225 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:33.225 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:33.225 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:12:33.225 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:12:33.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:12:33.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:12:33.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:12:33.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:12:33.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=4 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:12:33.325 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:33.513 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=6 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:12:33.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/6/type 2026-02-17T12:12:33.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=6 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.6 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.6 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.6 ']' 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:33.517 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:33.518 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:33.518 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.6.asok 2026-02-17T12:12:33.518 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:12:33.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:12:33.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:12:33.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:12:33.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.6 2026-02-17T12:12:33.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.6 2026-02-17T12:12:33.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.6 ']' 2026-02-17T12:12:33.566 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:33.567 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:33.567 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:33.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.6.asok 2026-02-17T12:12:33.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:12:33.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-17T12:12:33.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-17T12:12:33.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:12:33.616 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:12:33.616 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:12:33.616 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:33.807 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:7' 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:434: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:436: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:12:33.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:12:33.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:12:35.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:12:35.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:437: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-17T12:12:35.308 INFO:tasks.workunit.client.0.trial044.stderr:osd.7 is already down. 2026-02-17T12:12:35.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-17T12:12:35.561 INFO:tasks.workunit.client.0.trial044.stderr:osd.7 is already out. 2026-02-17T12:12:35.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-17T12:12:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:35.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:35.570 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:35.570 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:35.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:35.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:35.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:35.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:35.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:35.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:35.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:35.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:35.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:35.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:6 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:7 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:8' 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:35.843 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:35.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803796 2026-02-17T12:12:35.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803796 2026-02-17T12:12:35.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796' 2026-02-17T12:12:35.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:35.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:12:35.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574866 2026-02-17T12:12:35.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574866 2026-02-17T12:12:35.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866' 2026-02-17T12:12:35.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:35.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:12:36.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411346 2026-02-17T12:12:36.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411346 2026-02-17T12:12:36.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346' 2026-02-17T12:12:36.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.035 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:12:36.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149711 2026-02-17T12:12:36.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149711 2026-02-17T12:12:36.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346 3-111669149711' 2026-02-17T12:12:36.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.101 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:12:36.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888077 2026-02-17T12:12:36.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888077 2026-02-17T12:12:36.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346 3-111669149711 4-146028888077' 2026-02-17T12:12:36.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:12:36.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593741 2026-02-17T12:12:36.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593741 2026-02-17T12:12:36.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346 3-111669149711 4-146028888077 5-184683593741' 2026-02-17T12:12:36.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:12:36.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397515 2026-02-17T12:12:36.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397515 2026-02-17T12:12:36.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346 3-111669149711 4-146028888077 5-184683593741 6-210453397515' 2026-02-17T12:12:36.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T12:12:36.332 INFO:tasks.workunit.client.0.trial044.stderr:Error ENXIO: problem getting command descriptions from osd.7 2026-02-17T12:12:36.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T12:12:36.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T12:12:36.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T12:12:36.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:36.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906951 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906951 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803796 1-55834574866 2-77309411346 3-111669149711 4-146028888077 5-184683593741 6-210453397515 8-274877906951' 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803796 2026-02-17T12:12:36.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:36.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:36.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803796 2026-02-17T12:12:36.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:36.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803796 2026-02-17T12:12:36.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803796' 2026-02-17T12:12:36.396 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803796 2026-02-17T12:12:36.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:36.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803794 -lt 25769803796 2026-02-17T12:12:36.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:37.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:37.590 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:37.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803796 -lt 25769803796 2026-02-17T12:12:37.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:37.788 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-55834574866 2026-02-17T12:12:37.788 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:37.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:12:37.789 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-55834574866 2026-02-17T12:12:37.789 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:37.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574866 2026-02-17T12:12:37.790 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 55834574866 2026-02-17T12:12:37.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 55834574866' 2026-02-17T12:12:37.791 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:12:37.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574866 -lt 55834574866 2026-02-17T12:12:37.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:37.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411346 2026-02-17T12:12:37.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:37.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:12:37.980 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411346 2026-02-17T12:12:37.980 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:37.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411346 2026-02-17T12:12:37.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411346' 2026-02-17T12:12:37.982 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411346 2026-02-17T12:12:37.982 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411346 -lt 77309411346 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:38.173 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149711 2026-02-17T12:12:38.173 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149711 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:38.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149711 2026-02-17T12:12:38.175 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149711 2026-02-17T12:12:38.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149711' 2026-02-17T12:12:38.176 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:12:38.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149711 -lt 111669149711 2026-02-17T12:12:38.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:38.367 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888077 2026-02-17T12:12:38.367 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:38.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:12:38.368 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888077 2026-02-17T12:12:38.369 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:38.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888077 2026-02-17T12:12:38.370 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888077 2026-02-17T12:12:38.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888077' 2026-02-17T12:12:38.370 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:12:38.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888078 -lt 146028888077 2026-02-17T12:12:38.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:38.557 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-184683593741 2026-02-17T12:12:38.557 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:38.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:12:38.558 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-184683593741 2026-02-17T12:12:38.559 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:38.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593741 2026-02-17T12:12:38.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 184683593741' 2026-02-17T12:12:38.560 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 184683593741 2026-02-17T12:12:38.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:12:38.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593741 -lt 184683593741 2026-02-17T12:12:38.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:38.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-210453397515 2026-02-17T12:12:38.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:38.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:12:38.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-210453397515 2026-02-17T12:12:38.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:38.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397515 2026-02-17T12:12:38.751 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 210453397515 2026-02-17T12:12:38.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 210453397515' 2026-02-17T12:12:38.752 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:12:38.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397515 -lt 210453397515 2026-02-17T12:12:38.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:38.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-274877906951 2026-02-17T12:12:38.943 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:38.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T12:12:38.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-274877906951 2026-02-17T12:12:38.945 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:38.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906951 2026-02-17T12:12:38.946 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 274877906951' 2026-02-17T12:12:38.946 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.8 seq 274877906951 2026-02-17T12:12:38.947 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T12:12:39.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906951 -lt 274877906951 2026-02-17T12:12:39.135 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:39.136 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:39.136 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:39.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:12:39.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:39.388 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:39.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:39.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:39.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:39.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:39.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:39.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:12:39.574 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:39.574 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:39.574 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:39.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:12:39.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:39.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:39.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:12:39.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-17T12:12:39.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:12:39.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:12:39.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:445: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:12:39.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:12:39.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:12:40.039 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:12:40.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:12:40.249 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:12:40.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:12:40.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:12:40.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:12:40.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:12:40.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:12:40.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:12:40.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:12:40.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:12:40.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:12:40.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:12:40.368 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:12:40.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:12:40.369 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:12:40.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:12:40.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:12:40.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:12:40.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:40.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:12:40.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:12:40.372 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:40.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:12:40.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:12:40.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:12:40.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:12:40.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:12:40.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:12:40.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:12:40.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:12:40.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:12:40.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:12:40.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:12:40.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:12:40.398 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:12:40.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:12:40.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:12:40.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:12:40.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:40.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:12:40.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:12:40.401 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:40.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:12:40.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:12:40.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:12:40.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:12:40.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:12:40.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:12:40.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:12:40.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:12:40.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:12:40.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:12:40.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:12:40.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:12:40.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:12:40.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:12:40.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:12:40.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.429 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:40.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:12:40.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:12:40.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:12:40.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:12:40.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:12:40.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.457 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.458 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:12:40.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:12:40.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:12:40.502 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:12:40.503 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:12:40.503 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:40.504 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.504 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.504 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:12:40.504 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:12:40.504 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:12:40.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:12:40.644 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:12:40.644 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:40.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:40.645 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:40.645 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:40.645 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:40.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:40.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:12:40.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:12:40.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:12:40.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:12:40.760 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:12:40.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:41.769 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:12:41.769 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:12:41.769 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:12:41.769 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:41.769 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:41.770 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:41.770 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:12:41.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:12:41.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:12:41.810 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:12:41.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:12:41.815 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:12:40.446+0000 7f6946fd0d80 0 load: jerasure load: lrc 2026-02-17T12:12:41.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:452: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:455: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:12:41.817 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-17T12:12:41.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:12:41.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:41.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:12:41.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=68bedf48-8e59-4213-bb7b-1498172796c3 2026-02-17T12:12:41.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 68bedf48-8e59-4213-bb7b-1498172796c3' 2026-02-17T12:12:41.826 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 68bedf48-8e59-4213-bb7b-1498172796c3 2026-02-17T12:12:41.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:41.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA5W5RpZk+xMRAAE3Wq8psrCd6A0ko32EHH7Q== 2026-02-17T12:12:41.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA5W5RpZk+xMRAAE3Wq8psrCd6A0ko32EHH7Q=="}' 2026-02-17T12:12:41.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 68bedf48-8e59-4213-bb7b-1498172796c3 -i td/test-erasure-eio/0/new.json 2026-02-17T12:12:41.940 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:41.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:12:41.948 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5W5RpZk+xMRAAE3Wq8psrCd6A0ko32EHH7Q== --osd-uuid 68bedf48-8e59-4213-bb7b-1498172796c3 2026-02-17T12:12:41.968 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:41.965+0000 7f6af3e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.969 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:41.967+0000 7f6af3e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.970 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:41.968+0000 7f6af3e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.971 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:41.968+0000 7f6af3e2d780 -1 bdev(0x561b95537c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:41.971 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:41.969+0000 7f6af3e2d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:12:44.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:12:44.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:44.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:12:44.094 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:12:44.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:44.351 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:12:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:12:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:12:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:44.370 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:44.368+0000 7ff36d80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:44.371 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:44.369+0000 7ff36d80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:44.372 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:44.370+0000 7ff36d80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:44.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:44.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:44.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:45.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:45.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:45.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:45.740 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:45.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:45.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:45.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:45.957 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:45.954+0000 7ff36d80b780 -1 Falling back to public interface 2026-02-17T12:12:46.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:46.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:46.931 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:46.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:46.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:46.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:47.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:48.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:48.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:48.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:48.125 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:12:48.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:48.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:48.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:48.914 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:48.913+0000 7ff36d80b780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:12:49.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:49.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:49.325 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:12:49.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:49.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:49.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:49.517 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2056391879,v1:127.0.0.1:6803/2056391879] [v2:127.0.0.1:6804/2056391879,v1:127.0.0.1:6805/2056391879] exists,up 68bedf48-8e59-4213-bb7b-1498172796c3 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:49.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:49.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3c45962c-1430-4709-b91c-ab596edd5118 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3c45962c-1430-4709-b91c-ab596edd5118' 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 3c45962c-1430-4709-b91c-ab596edd5118 2026-02-17T12:12:49.521 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:49.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBBW5RpUyzNHxAAGltzm0x7BwZTzvbYrDMxJw== 2026-02-17T12:12:49.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBBW5RpUyzNHxAAGltzm0x7BwZTzvbYrDMxJw=="}' 2026-02-17T12:12:49.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3c45962c-1430-4709-b91c-ab596edd5118 -i td/test-erasure-eio/1/new.json 2026-02-17T12:12:49.728 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:49.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:12:49.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBW5RpUyzNHxAAGltzm0x7BwZTzvbYrDMxJw== --osd-uuid 3c45962c-1430-4709-b91c-ab596edd5118 2026-02-17T12:12:49.757 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:49.756+0000 7fa472815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:49.758 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:49.757+0000 7fa472815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:49.759 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:49.758+0000 7fa472815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:49.760 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:49.759+0000 7fa472815780 -1 bdev(0x55889f017c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:49.760 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:49.759+0000 7fa472815780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:12:52.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:12:52.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:52.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:12:52.138 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:12:52.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:52.397 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:12:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:12:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:12:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:52.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:52.417 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:52.415+0000 7f4c12153780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:52.418 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:52.417+0000 7f4c12153780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:52.419 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:52.418+0000 7f4c12153780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:12:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:12:52.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:52.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:52.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:52.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:52.589 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:12:52.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:52.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:52.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:53.779 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:12:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:53.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:54.002 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:54.001+0000 7f4c12153780 -1 Falling back to public interface 2026-02-17T12:12:54.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:54.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:54.972 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:54.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:54.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:54.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:55.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:56.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:56.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:56.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:56.170 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:12:56.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:56.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:56.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:56.963 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:56.962+0000 7f4c12153780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:12:57.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:57.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:57.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:57.363 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:12:57.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:57.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:57.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:58.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:58.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:58.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:58.552 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:12:58.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:58.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:58.744 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 11 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1799315888,v1:127.0.0.1:6811/1799315888] [v2:127.0.0.1:6812/1799315888,v1:127.0.0.1:6813/1799315888] exists,up 3c45962c-1430-4709-b91c-ab596edd5118 2026-02-17T12:12:58.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:58.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:58.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:58.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:58.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:12:58.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:12:58.752 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:58.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=66ef64d5-b4e6-4a9d-a743-663767d18667 2026-02-17T12:12:58.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 66ef64d5-b4e6-4a9d-a743-663767d18667' 2026-02-17T12:12:58.752 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 66ef64d5-b4e6-4a9d-a743-663767d18667 2026-02-17T12:12:58.752 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:58.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBKW5RpYk5QLRAAFxNkPm+9Ycv9p8pAVqgyEw== 2026-02-17T12:12:58.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBKW5RpYk5QLRAAFxNkPm+9Ycv9p8pAVqgyEw=="}' 2026-02-17T12:12:58.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 66ef64d5-b4e6-4a9d-a743-663767d18667 -i td/test-erasure-eio/2/new.json 2026-02-17T12:12:58.954 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:12:58.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:12:58.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKW5RpYk5QLRAAFxNkPm+9Ycv9p8pAVqgyEw== --osd-uuid 66ef64d5-b4e6-4a9d-a743-663767d18667 2026-02-17T12:12:58.981 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:58.980+0000 7f281b9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:58.982 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:58.981+0000 7f281b9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:58.983 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:58.982+0000 7f281b9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:58.984 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:58.983+0000 7f281b9d4780 -1 bdev(0x55fb60e5bc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:58.984 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:12:58.983+0000 7f281b9d4780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:13:01.098 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:13:01.098 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:01.099 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:13:01.099 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:13:01.100 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:01.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:13:01.356 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:13:01.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:01.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:01.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:01.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:01.375 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:01.373+0000 7fdfce920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:01.376 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:01.375+0000 7fdfce920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:01.377 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:01.376+0000 7fdfce920780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:01.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:01.547 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:01.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:02.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:02.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:02.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:02.738 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:02.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:02.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:02.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:02.962 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:02.960+0000 7fdfce920780 -1 Falling back to public interface 2026-02-17T12:13:03.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:03.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:03.929 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:03.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:03.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:03.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:04.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:05.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:05.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:05.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:05.121 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:05.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:05.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:05.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:05.909 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:05.908+0000 7fdfce920780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:13:06.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:06.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:06.312 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:06.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:06.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:07.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:07.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:07.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:07.505 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:13:07.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:07.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:07.697 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2010201704,v1:127.0.0.1:6819/2010201704] [v2:127.0.0.1:6820/2010201704,v1:127.0.0.1:6821/2010201704] exists,up 66ef64d5-b4e6-4a9d-a743-663767d18667 2026-02-17T12:13:07.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:07.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:07.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:07.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:07.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:07.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:13:07.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:07.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93 2026-02-17T12:13:07.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93' 2026-02-17T12:13:07.705 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93 2026-02-17T12:13:07.705 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:07.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTW5Rp+qV+KhAATrhtEUewf7hMpCR2MdvENA== 2026-02-17T12:13:07.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTW5Rp+qV+KhAATrhtEUewf7hMpCR2MdvENA=="}' 2026-02-17T12:13:07.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93 -i td/test-erasure-eio/3/new.json 2026-02-17T12:13:07.908 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:07.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:13:07.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTW5Rp+qV+KhAATrhtEUewf7hMpCR2MdvENA== --osd-uuid 35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93 2026-02-17T12:13:07.936 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:07.935+0000 7f09d4413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.938 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:07.936+0000 7f09d4413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.938 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:07.937+0000 7f09d4413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.939 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:07.938+0000 7f09d4413780 -1 bdev(0x5576a6b8bc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:07.939 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:07.938+0000 7f09d4413780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:13:10.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:13:10.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:10.059 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:13:10.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:13:10.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:10.318 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:13:10.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:13:10.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:10.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:10.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:10.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:10.337 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:10.335+0000 7f0696e23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:10.338 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:10.337+0000 7f0696e23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:10.339 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:10.338+0000 7f0696e23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:13:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:10.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:13:10.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:10.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:10.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:10.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:10.510 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:10.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:10.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:11.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:11.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:11.699 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:11.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:11.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:11.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:11.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:11.927 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:11.926+0000 7f0696e23780 -1 Falling back to public interface 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:12.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:13.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:14.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:14.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:14.093 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:14.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:14.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:14.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:14.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:14.889 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:14.888+0000 7f0696e23780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:13:15.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:15.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:15.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:15.285 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:15.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:15.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:15.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:16.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:16.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:16.477 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:13:16.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:16.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:16.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:16.667 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2792761994,v1:127.0.0.1:6827/2792761994] [v2:127.0.0.1:6828/2792761994,v1:127.0.0.1:6829/2792761994] exists,up 35d1d36d-0f92-4faf-b2d4-e80ac0dc4f93 2026-02-17T12:13:16.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:16.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:16.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:16.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3f70ca2b-c813-4410-be95-7ac820403a7b 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 3f70ca2b-c813-4410-be95-7ac820403a7b' 2026-02-17T12:13:16.674 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 3f70ca2b-c813-4410-be95-7ac820403a7b 2026-02-17T12:13:16.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:16.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBcW5Rpltu0KBAAkTkVruRSzRXTTzx6/vHhDg== 2026-02-17T12:13:16.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBcW5Rpltu0KBAAkTkVruRSzRXTTzx6/vHhDg=="}' 2026-02-17T12:13:16.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3f70ca2b-c813-4410-be95-7ac820403a7b -i td/test-erasure-eio/4/new.json 2026-02-17T12:13:16.879 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:16.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:13:16.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcW5Rpltu0KBAAkTkVruRSzRXTTzx6/vHhDg== --osd-uuid 3f70ca2b-c813-4410-be95-7ac820403a7b 2026-02-17T12:13:16.906 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:16.905+0000 7fb53cbd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.907 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:16.906+0000 7fb53cbd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.908 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:16.907+0000 7fb53cbd0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.909 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:16.908+0000 7fb53cbd0780 -1 bdev(0x55f4e3241c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:16.909 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:16.908+0000 7fb53cbd0780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:13:19.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:13:19.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:19.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:13:19.032 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:13:19.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:19.300 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:13:19.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:13:19.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:19.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:19.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:19.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:19.318 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:19.316+0000 7f1d2980d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:19.320 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:19.318+0000 7f1d2980d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:19.321 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:19.319+0000 7f1d2980d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:19.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:19.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:19.491 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:19.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:19.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:13:19.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:20.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:20.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:20.686 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:20.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:20.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:20.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:13:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:21.151 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:21.149+0000 7f1d2980d780 -1 Falling back to public interface 2026-02-17T12:13:21.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:21.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:21.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:21.877 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:21.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:21.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:13:22.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:23.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:23.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:23.066 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:23.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:23.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:23.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:13:23.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:23.858 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:23.856+0000 7f1d2980d780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:13:24.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:24.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:24.258 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:24.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:24.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:24.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:13:24.456 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1673470583,v1:127.0.0.1:6835/1673470583] [v2:127.0.0.1:6836/1673470583,v1:127.0.0.1:6837/1673470583] exists,up 3f70ca2b-c813-4410-be95-7ac820403a7b 2026-02-17T12:13:24.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:24.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:24.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:24.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:24.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:24.462 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:24.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9fd29bd4-1b7f-4426-942d-5bb443cb95b5 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 9fd29bd4-1b7f-4426-942d-5bb443cb95b5' 2026-02-17T12:13:24.464 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 9fd29bd4-1b7f-4426-942d-5bb443cb95b5 2026-02-17T12:13:24.465 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:24.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBkW5Rp4O4kHBAArGgV42hsdB1YnVHbPkFXPA== 2026-02-17T12:13:24.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBkW5Rp4O4kHBAArGgV42hsdB1YnVHbPkFXPA=="}' 2026-02-17T12:13:24.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9fd29bd4-1b7f-4426-942d-5bb443cb95b5 -i td/test-erasure-eio/5/new.json 2026-02-17T12:13:24.666 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:13:24.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-17T12:13:24.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBkW5Rp4O4kHBAArGgV42hsdB1YnVHbPkFXPA== --osd-uuid 9fd29bd4-1b7f-4426-942d-5bb443cb95b5 2026-02-17T12:13:24.694 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:24.692+0000 7fe2d383f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:24.695 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:24.693+0000 7fe2d383f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:24.696 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:24.694+0000 7fe2d383f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:24.697 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:24.695+0000 7fe2d383f780 -1 bdev(0x55db31bafc00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:24.697 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:24.695+0000 7fe2d383f780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-17T12:13:26.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-17T12:13:26.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:26.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:13:26.816 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:13:26.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:27.070 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:13:27.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:13:27.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:27.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:27.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:27.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:27.089 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:27.088+0000 7f705ab3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:27.090 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:27.089+0000 7f705ab3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:27.091 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:27.090+0000 7f705ab3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:27.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:27.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:27.261 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:27.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:27.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:13:27.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:28.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:28.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:28.449 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:28.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:28.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:28.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:13:28.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:28.681 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:28.680+0000 7f705ab3f780 -1 Falling back to public interface 2026-02-17T12:13:29.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:29.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:29.643 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:29.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:29.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:29.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:13:29.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:30.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:30.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:30.837 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:30.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:30.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:30.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:13:31.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:31.622 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:31.620+0000 7f705ab3f780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:13:32.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:32.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:32.029 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:32.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:32.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:32.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:13:32.223 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/519486439,v1:127.0.0.1:6843/519486439] [v2:127.0.0.1:6844/519486439,v1:127.0.0.1:6845/519486439] exists,up 9fd29bd4-1b7f-4426-942d-5bb443cb95b5 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:32.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:32.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-17T12:13:32.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:32.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b3587140-ab7b-4050-9b9c-73cdcaa530a8 2026-02-17T12:13:32.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 b3587140-ab7b-4050-9b9c-73cdcaa530a8' 2026-02-17T12:13:32.227 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 b3587140-ab7b-4050-9b9c-73cdcaa530a8 2026-02-17T12:13:32.227 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:32.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBsW5RpQgRWDhAAxxhMTJfQK0TiyotEqXLq5A== 2026-02-17T12:13:32.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBsW5RpQgRWDhAAxxhMTJfQK0TiyotEqXLq5A=="}' 2026-02-17T12:13:32.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b3587140-ab7b-4050-9b9c-73cdcaa530a8 -i td/test-erasure-eio/6/new.json 2026-02-17T12:13:32.440 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:13:32.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-17T12:13:32.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBsW5RpQgRWDhAAxxhMTJfQK0TiyotEqXLq5A== --osd-uuid b3587140-ab7b-4050-9b9c-73cdcaa530a8 2026-02-17T12:13:32.469 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:32.467+0000 7f1ec9a09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:32.470 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:32.469+0000 7f1ec9a09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:32.471 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:32.470+0000 7f1ec9a09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:32.471 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:32.470+0000 7f1ec9a09780 -1 bdev(0x55e91e7b9c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:32.471 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:32.470+0000 7f1ec9a09780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-17T12:13:34.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-17T12:13:34.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:34.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:13:34.593 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:13:34.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:34.848 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:13:34.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:13:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:34.868 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:34.866+0000 7f6fa601c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:34.869 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:34.868+0000 7f6fa601c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:34.870 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:34.868+0000 7f6fa601c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:35.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:35.040 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:35.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:35.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:35.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:36.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:36.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:36.230 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:36.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:36.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:36.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:36.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:36.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:36.458+0000 7f6fa601c780 -1 Falling back to public interface 2026-02-17T12:13:37.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:37.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:37.424 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:37.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:37.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:37.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:37.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:38.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:38.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:38.614 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:38.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:38.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:38.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:38.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:39.418 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:39.417+0000 7f6fa601c780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:13:39.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:39.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:39.807 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:13:39.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:39.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:39.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:39.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:41.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:41.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:41.001 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:13:41.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:41.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:41.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:13:41.190 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2803007225,v1:127.0.0.1:6851/2803007225] [v2:127.0.0.1:6852/2803007225,v1:127.0.0.1:6853/2803007225] exists,up b3587140-ab7b-4050-9b9c-73cdcaa530a8 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:13:41.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:41.193 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:41.193 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:41.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:13:41.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:13:41.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:13:41.237 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:13:41.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:13:41.244 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:12:45.957+0000 7ff36d80b780 0 load: jerasure load: lrc 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:459: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:460: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:13:41.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:13:41.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:13:41.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:13:41.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:13:41.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:13:41.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:13:41.788 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:13:41.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:13:42.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:13:42.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:13:42.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:13:42.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:13:42.798 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:13:42.799 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:13:42.799 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:13:42.799 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:13:42.799 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:13:42.799 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:13:42.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:13:42.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:13:42.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:13:42.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:13:42.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:13:42.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:13:43.070 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:13:43.071 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:13:43.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:13:43.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:13:43.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:13:43.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:13:43.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:13:43.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.134 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:13:43.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=47244640269 2026-02-17T12:13:43.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 47244640269 2026-02-17T12:13:43.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269' 2026-02-17T12:13:43.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:13:43.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411339 2026-02-17T12:13:43.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411339 2026-02-17T12:13:43.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269 2-77309411339' 2026-02-17T12:13:43.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.260 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:13:43.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117000 2026-02-17T12:13:43.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117000 2026-02-17T12:13:43.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269 2-77309411339 3-115964117000' 2026-02-17T12:13:43.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:13:43.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855366 2026-02-17T12:13:43.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855366 2026-02-17T12:13:43.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269 2-77309411339 3-115964117000 4-150323855366' 2026-02-17T12:13:43.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:13:43.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626437 2026-02-17T12:13:43.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626437 2026-02-17T12:13:43.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269 2-77309411339 3-115964117000 4-150323855366 5-180388626437' 2026-02-17T12:13:43.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:43.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:13:43.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397508 2026-02-17T12:13:43.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397508 2026-02-17T12:13:43.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-47244640269 2-77309411339 3-115964117000 4-150323855366 5-180388626437 6-210453397508' 2026-02-17T12:13:43.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:43.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:13:43.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:43.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:13:43.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:13:43.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:43.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:13:43.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:13:43.515 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803789 2026-02-17T12:13:43.516 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:43.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:13:43.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:13:44.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:13:44.707 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:44.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:13:44.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:44.897 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-47244640269 2026-02-17T12:13:44.897 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:44.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:13:44.898 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-47244640269 2026-02-17T12:13:44.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:44.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=47244640269 2026-02-17T12:13:44.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 47244640269' 2026-02-17T12:13:44.900 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 47244640269 2026-02-17T12:13:44.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:13:45.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 47244640269 -lt 47244640269 2026-02-17T12:13:45.090 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:45.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411339 2026-02-17T12:13:45.091 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:45.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:13:45.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411339 2026-02-17T12:13:45.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:45.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411339 2026-02-17T12:13:45.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411339' 2026-02-17T12:13:45.094 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411339 2026-02-17T12:13:45.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:13:45.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411339 -lt 77309411339 2026-02-17T12:13:45.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:45.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-115964117000 2026-02-17T12:13:45.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:45.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:13:45.284 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-115964117000 2026-02-17T12:13:45.284 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:45.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117000 2026-02-17T12:13:45.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 115964117000' 2026-02-17T12:13:45.285 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 115964117000 2026-02-17T12:13:45.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:13:45.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117000 -lt 115964117000 2026-02-17T12:13:45.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:45.473 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855366 2026-02-17T12:13:45.473 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:45.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:13:45.475 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855366 2026-02-17T12:13:45.475 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:45.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855366 2026-02-17T12:13:45.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855366' 2026-02-17T12:13:45.476 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855366 2026-02-17T12:13:45.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:13:45.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855366 2026-02-17T12:13:45.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:45.667 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626437 2026-02-17T12:13:45.667 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:45.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:13:45.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626437 2026-02-17T12:13:45.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:45.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626437 2026-02-17T12:13:45.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626437' 2026-02-17T12:13:45.671 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626437 2026-02-17T12:13:45.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:13:45.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626437 -lt 180388626437 2026-02-17T12:13:45.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:45.862 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-210453397508 2026-02-17T12:13:45.862 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:45.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:13:45.864 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-210453397508 2026-02-17T12:13:45.864 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:45.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397508 2026-02-17T12:13:45.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 210453397508' 2026-02-17T12:13:45.865 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 210453397508 2026-02-17T12:13:45.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:13:46.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397508 -lt 210453397508 2026-02-17T12:13:46.055 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:13:46.055 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:46.055 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:46.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:13:46.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:13:46.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:13:46.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:13:46.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:13:46.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:46.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:462: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:13:46.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:13:46.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:13:46.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:13:46.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-17T12:13:46.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:13:46.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:13:46.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:13:46.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:464: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:13:46.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-17T12:13:46.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:466: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-17T12:13:47.049 INFO:tasks.workunit.client.0.trial044.stderr:osd.0 is already out. 2026-02-17T12:13:47.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:469: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:13:47.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:13:47.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:13:47.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:13:47.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:13:47.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:13:47.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:13:47.142 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:13:47.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:13:47.333 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:13:47.333 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:13:47.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803792 2026-02-17T12:13:47.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803792 2026-02-17T12:13:47.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792' 2026-02-17T12:13:47.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:13:47.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=47244640271 2026-02-17T12:13:47.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 47244640271 2026-02-17T12:13:47.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271' 2026-02-17T12:13:47.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:13:47.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411342 2026-02-17T12:13:47.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411342 2026-02-17T12:13:47.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271 2-77309411342' 2026-02-17T12:13:47.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.521 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:13:47.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117003 2026-02-17T12:13:47.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117003 2026-02-17T12:13:47.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271 2-77309411342 3-115964117003' 2026-02-17T12:13:47.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.584 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:13:47.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855369 2026-02-17T12:13:47.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855369 2026-02-17T12:13:47.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271 2-77309411342 3-115964117003 4-150323855369' 2026-02-17T12:13:47.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:13:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626439 2026-02-17T12:13:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626439 2026-02-17T12:13:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271 2-77309411342 3-115964117003 4-150323855369 5-180388626439' 2026-02-17T12:13:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:47.710 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:13:47.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397511 2026-02-17T12:13:47.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397511 2026-02-17T12:13:47.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-47244640271 2-77309411342 3-115964117003 4-150323855369 5-180388626439 6-210453397511' 2026-02-17T12:13:47.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:47.773 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803792 2026-02-17T12:13:47.773 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:47.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:13:47.774 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803792 2026-02-17T12:13:47.774 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:47.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803792 2026-02-17T12:13:47.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-17T12:13:47.776 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803792 2026-02-17T12:13:47.776 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:47.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-17T12:13:47.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:13:48.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:13:48.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:49.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-17T12:13:49.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:49.161 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-47244640271 2026-02-17T12:13:49.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:49.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:13:49.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-47244640271 2026-02-17T12:13:49.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=47244640271 2026-02-17T12:13:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 47244640271' 2026-02-17T12:13:49.164 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 47244640271 2026-02-17T12:13:49.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:13:49.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 47244640272 -lt 47244640271 2026-02-17T12:13:49.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:49.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411342 2026-02-17T12:13:49.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:49.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:13:49.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411342 2026-02-17T12:13:49.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:49.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411342 2026-02-17T12:13:49.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-17T12:13:49.353 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411342 2026-02-17T12:13:49.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:13:49.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411342 -lt 77309411342 2026-02-17T12:13:49.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:49.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-115964117003 2026-02-17T12:13:49.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:49.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:13:49.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-115964117003 2026-02-17T12:13:49.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:49.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117003 2026-02-17T12:13:49.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 115964117003' 2026-02-17T12:13:49.546 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 115964117003 2026-02-17T12:13:49.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:13:49.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117003 -lt 115964117003 2026-02-17T12:13:49.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:49.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-150323855369 2026-02-17T12:13:49.735 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:49.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:13:49.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-150323855369 2026-02-17T12:13:49.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:49.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855369 2026-02-17T12:13:49.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 150323855369' 2026-02-17T12:13:49.738 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 150323855369 2026-02-17T12:13:49.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:13:49.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855369 2026-02-17T12:13:49.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:49.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626439 2026-02-17T12:13:49.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:49.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:13:49.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626439 2026-02-17T12:13:49.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:49.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626439 2026-02-17T12:13:49.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626439' 2026-02-17T12:13:49.931 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626439 2026-02-17T12:13:49.931 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:13:50.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626440 -lt 180388626439 2026-02-17T12:13:50.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:50.121 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-210453397511 2026-02-17T12:13:50.121 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:50.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:13:50.122 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-210453397511 2026-02-17T12:13:50.122 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:50.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397511 2026-02-17T12:13:50.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 210453397511' 2026-02-17T12:13:50.124 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 210453397511 2026-02-17T12:13:50.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:13:50.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397511 -lt 210453397511 2026-02-17T12:13:50.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:13:50.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:50.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:50.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:13:50.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:13:50.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:13:50.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:13:50.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:13:50.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:13:50.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:13:50.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:13:50.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:13:50.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:13:50.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:50.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:51.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:13:51.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:13:51.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-17T12:13:51.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:13:51.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:13:51.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:13:51.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-17T12:13:51.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:13:51.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:473: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:13:51.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-17T12:13:51.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:13:51.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:13:51.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:13:51.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:13:51.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:13:51.301 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:13:51.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:13:51.508 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:13:51.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:13:51.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:13:51.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:13:51.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:13:51.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:51.517 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:51.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:51.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:51.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:51.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:51.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:13:51.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:13:51.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:13:51.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:13:51.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:13:51.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:13:51.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:51.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:13:51.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:13:51.634 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:51.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:13:51.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:13:51.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:13:51.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:13:51.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:13:51.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:13:51.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:51.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:51.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:51.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:51.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:13:51.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:13:51.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:13:51.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:13:51.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:13:51.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:13:51.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:51.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:13:51.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:13:51.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:51.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:13:51.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:13:51.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:13:51.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:13:51.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:13:51.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:13:51.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:13:51.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:13:51.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:13:51.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:13:51.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:13:51.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:13:51.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:51.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:51.686 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:51.686 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.686 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:51.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:13:51.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:13:51.713 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:13:51.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:13:51.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:13:51.760 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:13:51.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:13:51.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:13:51.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:13:51.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:13:51.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:13:51.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:13:51.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:51.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:51.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:13:51.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:13:51.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:13:51.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:13:52.018 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:13:52.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:53.027 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:53.028 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:13:53.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:13:53.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:13:53.068 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:13:53.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:13:53.074 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:13:51.703+0000 7f0db77cad80 0 load: jerasure load: lrc 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:481: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:485: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:13:53.075 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-17T12:13:53.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:13:53.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:13:53.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e0699bc5-7c00-4fcf-a2b2-ed3a964145b2 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 e0699bc5-7c00-4fcf-a2b2-ed3a964145b2' 2026-02-17T12:13:53.080 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 e0699bc5-7c00-4fcf-a2b2-ed3a964145b2 2026-02-17T12:13:53.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:53.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCBW5RpOpCBBRAARHWQ3XuWAFIcf2LJdS+TMg== 2026-02-17T12:13:53.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCBW5RpOpCBBRAARHWQ3XuWAFIcf2LJdS+TMg=="}' 2026-02-17T12:13:53.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e0699bc5-7c00-4fcf-a2b2-ed3a964145b2 -i td/test-erasure-eio/0/new.json 2026-02-17T12:13:53.198 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:53.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:13:53.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBW5RpOpCBBRAARHWQ3XuWAFIcf2LJdS+TMg== --osd-uuid e0699bc5-7c00-4fcf-a2b2-ed3a964145b2 2026-02-17T12:13:53.225 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:53.224+0000 7fe8f2bd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:53.227 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:53.226+0000 7fe8f2bd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:53.228 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:53.226+0000 7fe8f2bd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:53.228 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:53.227+0000 7fe8f2bd4780 -1 bdev(0x55bce39b7c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:53.228 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:53.227+0000 7fe8f2bd4780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:13:55.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:13:55.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:55.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:13:55.608 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:13:55.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:55.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:13:55.864 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:13:55.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:55.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:55.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:55.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:55.882 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:55.881+0000 7f6778e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:55.883 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:55.882+0000 7f6778e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:55.884 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:55.883+0000 7f6778e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:56.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:13:56.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:56.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:13:56.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:56.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:56.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:56.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:56.057 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:13:56.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:56.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:56.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:57.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:57.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:57.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:57.247 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:13:57.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:57.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:57.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:57.472 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:13:57.470+0000 7f6778e0e780 -1 Falling back to public interface 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:58.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:58.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:59.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:59.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:59.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:59.632 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:13:59.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:59.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:59.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:00.410 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:00.408+0000 7f6778e0e780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:14:00.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:00.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:00.824 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:00.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:00.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:00.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:02.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:02.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:02.024 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:14:02.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:02.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:02.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:02.217 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/590371027,v1:127.0.0.1:6803/590371027] [v2:127.0.0.1:6804/590371027,v1:127.0.0.1:6805/590371027] exists,up e0699bc5-7c00-4fcf-a2b2-ed3a964145b2 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:14:02.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:02.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:02.220 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:02.220 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:02.220 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:14:02.221 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:02.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=366307d2-672a-4df2-bcfc-7ad7c678eae5 2026-02-17T12:14:02.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 366307d2-672a-4df2-bcfc-7ad7c678eae5' 2026-02-17T12:14:02.222 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 366307d2-672a-4df2-bcfc-7ad7c678eae5 2026-02-17T12:14:02.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:02.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCKW5RpafvpDRAAPDjMvIgdMTm8ejkY1QjgKw== 2026-02-17T12:14:02.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCKW5RpafvpDRAAPDjMvIgdMTm8ejkY1QjgKw=="}' 2026-02-17T12:14:02.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 366307d2-672a-4df2-bcfc-7ad7c678eae5 -i td/test-erasure-eio/1/new.json 2026-02-17T12:14:02.426 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:02.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:14:02.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKW5RpafvpDRAAPDjMvIgdMTm8ejkY1QjgKw== --osd-uuid 366307d2-672a-4df2-bcfc-7ad7c678eae5 2026-02-17T12:14:02.454 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:02.451+0000 7f8b3d934780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:02.455 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:02.452+0000 7f8b3d934780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:02.456 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:02.453+0000 7f8b3d934780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:02.456 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:02.454+0000 7f8b3d934780 -1 bdev(0x55cf3d3a5c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:02.457 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:02.454+0000 7f8b3d934780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:14:04.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:14:04.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:04.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:14:04.578 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:14:04.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:04.833 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:14:04.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:14:04.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:04.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:04.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:04.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:04.852 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:04.849+0000 7f1de6eed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:04.853 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:04.851+0000 7f1de6eed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:04.854 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:04.852+0000 7f1de6eed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:05.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:05.031 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:05.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:05.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:05.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:05.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:06.204 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:06.202+0000 7f1de6eed780 -1 Falling back to public interface 2026-02-17T12:14:06.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:06.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:06.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:06.222 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:06.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:06.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:06.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:07.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:07.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:08.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:08.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:08.914 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:08.913+0000 7f1de6eed780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:09.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:09.998 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3212432494,v1:127.0.0.1:6811/3212432494] [v2:127.0.0.1:6812/3212432494,v1:127.0.0.1:6813/3212432494] exists,up 366307d2-672a-4df2-bcfc-7ad7c678eae5 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:09.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:10.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:14:10.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:10.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=19f8ec96-ef53-4272-ad73-1fcefda37caf 2026-02-17T12:14:10.001 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 19f8ec96-ef53-4272-ad73-1fcefda37caf 2026-02-17T12:14:10.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 19f8ec96-ef53-4272-ad73-1fcefda37caf' 2026-02-17T12:14:10.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:10.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCSW5RpGhrZABAAJ5jWI3eJ5rfmdh2rMFQKHA== 2026-02-17T12:14:10.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCSW5RpGhrZABAAJ5jWI3eJ5rfmdh2rMFQKHA=="}' 2026-02-17T12:14:10.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 19f8ec96-ef53-4272-ad73-1fcefda37caf -i td/test-erasure-eio/2/new.json 2026-02-17T12:14:10.211 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:10.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:14:10.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSW5RpGhrZABAAJ5jWI3eJ5rfmdh2rMFQKHA== --osd-uuid 19f8ec96-ef53-4272-ad73-1fcefda37caf 2026-02-17T12:14:10.238 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:10.237+0000 7fb92c0b8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:10.240 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:10.239+0000 7fb92c0b8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:10.240 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:10.239+0000 7fb92c0b8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:10.241 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:10.240+0000 7fb92c0b8780 -1 bdev(0x55f92954dc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:10.241 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:10.240+0000 7fb92c0b8780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:14:12.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:14:12.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:12.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:14:12.605 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:14:12.605 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:12.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:14:12.860 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:14:12.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:12.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:12.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:12.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:12.878 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:12.877+0000 7fe12fa2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:12.880 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:12.879+0000 7fe12fa2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:12.881 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:12.879+0000 7fe12fa2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:13.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:14:13.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:13.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:14:13.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:13.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:13.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:13.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:13.051 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:13.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:13.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:13.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:14.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:14.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:14.241 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:14.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:14.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:14.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:14.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:14.473 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:14.471+0000 7fe12fa2e780 -1 Falling back to public interface 2026-02-17T12:14:15.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:15.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:15.437 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:15.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:15.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:15.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:15.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:16.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:16.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:17.429 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:17.428+0000 7fe12fa2e780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:14:17.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:17.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:17.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:17.841 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:17.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:17.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:18.040 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3914138797,v1:127.0.0.1:6819/3914138797] [v2:127.0.0.1:6820/3914138797,v1:127.0.0.1:6821/3914138797] exists,up 19f8ec96-ef53-4272-ad73-1fcefda37caf 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:18.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:18.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:14:18.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:18.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde 2026-02-17T12:14:18.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde' 2026-02-17T12:14:18.044 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde 2026-02-17T12:14:18.044 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:18.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCaW5RpJ+9lAxAACHck25N8D8DhD+KZzGMKcw== 2026-02-17T12:14:18.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCaW5RpJ+9lAxAACHck25N8D8DhD+KZzGMKcw=="}' 2026-02-17T12:14:18.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde -i td/test-erasure-eio/3/new.json 2026-02-17T12:14:18.253 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:18.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:14:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCaW5RpJ+9lAxAACHck25N8D8DhD+KZzGMKcw== --osd-uuid b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde 2026-02-17T12:14:18.282 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:18.280+0000 7fe3c2a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:18.283 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:18.282+0000 7fe3c2a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:18.284 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:18.283+0000 7fe3c2a14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:18.284 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:18.283+0000 7fe3c2a14780 -1 bdev(0x5654d9ffdc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:18.284 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:18.283+0000 7fe3c2a14780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:14:20.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:14:20.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:20.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:14:20.395 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:14:20.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:20.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:14:20.655 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:14:20.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:20.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:20.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:20.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:20.672 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:20.671+0000 7f8364c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:20.674 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:20.672+0000 7f8364c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:20.674 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:20.673+0000 7f8364c10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:20.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:21.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:21.759 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:21.758+0000 7f8364c10780 -1 Falling back to public interface 2026-02-17T12:14:22.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:22.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:22.039 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:22.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:22.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:22.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:22.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:23.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:23.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:23.232 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:23.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:23.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:23.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:23.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:24.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:24.615 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:24.724 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:24.722+0000 7f8364c10780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:14:25.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:25.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:25.617 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:25.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:25.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:25.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:25.815 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2750872923,v1:127.0.0.1:6827/2750872923] [v2:127.0.0.1:6828/2750872923,v1:127.0.0.1:6829/2750872923] exists,up b8e2a2cd-7fe3-4f7a-8d6c-419f8e6a0dde 2026-02-17T12:14:25.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:25.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:25.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:25.821 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:25.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:25.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:14:25.824 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:25.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1be20e81-7590-41ed-9f32-a043d0d56d38 2026-02-17T12:14:25.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 1be20e81-7590-41ed-9f32-a043d0d56d38' 2026-02-17T12:14:25.824 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 1be20e81-7590-41ed-9f32-a043d0d56d38 2026-02-17T12:14:25.825 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:25.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQChW5RpXvmKMRAAKznRw+HL93wUqAkvlgDy+Q== 2026-02-17T12:14:25.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQChW5RpXvmKMRAAKznRw+HL93wUqAkvlgDy+Q=="}' 2026-02-17T12:14:25.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1be20e81-7590-41ed-9f32-a043d0d56d38 -i td/test-erasure-eio/4/new.json 2026-02-17T12:14:26.031 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:26.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:14:26.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChW5RpXvmKMRAAKznRw+HL93wUqAkvlgDy+Q== --osd-uuid 1be20e81-7590-41ed-9f32-a043d0d56d38 2026-02-17T12:14:26.059 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:26.058+0000 7fadfa9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:26.060 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:26.059+0000 7fadfa9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:26.061 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:26.060+0000 7fadfa9d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:26.061 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:26.060+0000 7fadfa9d4780 -1 bdev(0x55f03fef1c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:26.061 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:26.060+0000 7fadfa9d4780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:14:28.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:14:28.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:28.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:14:28.439 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:14:28.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:28.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:14:28.699 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:14:28.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:28.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:28.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:28.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:28.717 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:28.715+0000 7fb5cd6fe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:28.718 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:28.717+0000 7fb5cd6fe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:28.719 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:28.718+0000 7fb5cd6fe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:28.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:14:28.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:28.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:14:28.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:28.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:28.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:28.891 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:28.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:28.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:28.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:29.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:30.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:30.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:30.078 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:30.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:30.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:30.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:30.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:30.306 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:30.305+0000 7fb5cd6fe780 -1 Falling back to public interface 2026-02-17T12:14:31.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:31.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:31.271 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:31.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:31.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:31.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:31.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:32.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:32.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:32.462 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:32.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:32.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:32.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:32.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:33.254 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:33.253+0000 7fb5cd6fe780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:14:33.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:33.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:33.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:33.653 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:33.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:33.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:33.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:34.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:34.850 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:14:34.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:34.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:34.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:14:35.042 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/375130964,v1:127.0.0.1:6835/375130964] [v2:127.0.0.1:6836/375130964,v1:127.0.0.1:6837/375130964] exists,up 1be20e81-7590-41ed-9f32-a043d0d56d38 2026-02-17T12:14:35.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:35.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:35.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:35.048 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:35.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=74770551-c96c-41e4-83b8-fdb6f6e1b183 2026-02-17T12:14:35.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 74770551-c96c-41e4-83b8-fdb6f6e1b183' 2026-02-17T12:14:35.051 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 74770551-c96c-41e4-83b8-fdb6f6e1b183 2026-02-17T12:14:35.051 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:35.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCrW5RpDAByAxAAougf5Y+poiobEzghU+KyDQ== 2026-02-17T12:14:35.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCrW5RpDAByAxAAougf5Y+poiobEzghU+KyDQ=="}' 2026-02-17T12:14:35.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 74770551-c96c-41e4-83b8-fdb6f6e1b183 -i td/test-erasure-eio/5/new.json 2026-02-17T12:14:35.253 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:14:35.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-17T12:14:35.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCrW5RpDAByAxAAougf5Y+poiobEzghU+KyDQ== --osd-uuid 74770551-c96c-41e4-83b8-fdb6f6e1b183 2026-02-17T12:14:35.279 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:35.278+0000 7f74b9e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:35.281 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:35.280+0000 7f74b9e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:35.282 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:35.281+0000 7f74b9e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:35.282 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:35.281+0000 7f74b9e0d780 -1 bdev(0x5587ee199c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:35.282 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:35.281+0000 7f74b9e0d780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-17T12:14:37.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-17T12:14:37.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:37.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:14:37.417 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:14:37.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:37.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:37.690 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:37.689+0000 7f391242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:37.691 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:37.690+0000 7f391242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:37.692 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:37.691+0000 7f391242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:37.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:37.865 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:37.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:37.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:38.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:39.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:39.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:39.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:39.052 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:39.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:39.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:39.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:39.275 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:39.273+0000 7f391242c780 -1 Falling back to public interface 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:40.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:40.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:41.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:41.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:41.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:41.441 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:41.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:41.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:41.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:42.224 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:42.222+0000 7f391242c780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:14:42.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:42.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:42.632 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:42.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:42.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:42.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:42.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:43.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:43.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:43.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:43.828 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:14:43.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:43.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:14:44.021 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3384103499,v1:127.0.0.1:6843/3384103499] [v2:127.0.0.1:6844/3384103499,v1:127.0.0.1:6845/3384103499] exists,up 74770551-c96c-41e4-83b8-fdb6f6e1b183 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:14:44.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:44.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-17T12:14:44.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:44.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=53d62d59-f85a-4832-9fc4-6bb603400258 2026-02-17T12:14:44.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 53d62d59-f85a-4832-9fc4-6bb603400258' 2026-02-17T12:14:44.025 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 53d62d59-f85a-4832-9fc4-6bb603400258 2026-02-17T12:14:44.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:44.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC0W5RpvDJBAhAAzdj/x4T7WZuY7rWJlKqzKQ== 2026-02-17T12:14:44.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC0W5RpvDJBAhAAzdj/x4T7WZuY7rWJlKqzKQ=="}' 2026-02-17T12:14:44.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 53d62d59-f85a-4832-9fc4-6bb603400258 -i td/test-erasure-eio/6/new.json 2026-02-17T12:14:44.240 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:14:44.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-17T12:14:44.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC0W5RpvDJBAhAAzdj/x4T7WZuY7rWJlKqzKQ== --osd-uuid 53d62d59-f85a-4832-9fc4-6bb603400258 2026-02-17T12:14:44.272 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:44.269+0000 7f4cee03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:44.273 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:44.271+0000 7f4cee03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:44.274 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:44.272+0000 7f4cee03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:44.275 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:44.272+0000 7f4cee03f780 -1 bdev(0x55bd7ae4bc00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:44.275 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:44.272+0000 7f4cee03f780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-17T12:14:46.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-17T12:14:46.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:46.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:14:46.649 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:14:46.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:46.915 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:14:46.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:14:46.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:46.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:46.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:46.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:46.935 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:46.933+0000 7f57cf222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:46.937 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:46.935+0000 7f57cf222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:46.938 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:46.936+0000 7f57cf222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:47.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:14:47.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:47.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:14:47.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:47.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:47.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:47.107 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:14:47.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:47.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:47.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:14:47.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:48.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:48.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:48.294 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:14:48.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:48.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:48.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:14:48.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:48.519 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:48.517+0000 7f57cf222780 -1 Falling back to public interface 2026-02-17T12:14:49.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:49.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:49.486 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:14:49.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:49.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:49.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:14:49.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:50.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:14:50.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:51.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:14:51.459+0000 7f57cf222780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:14:51.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:51.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:51.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:51.867 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:14:51.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:51.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:14:52.066 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1331766957,v1:127.0.0.1:6851/1331766957] [v2:127.0.0.1:6852/1331766957,v1:127.0.0.1:6853/1331766957] exists,up 53d62d59-f85a-4832-9fc4-6bb603400258 2026-02-17T12:14:52.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:14:52.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:14:52.113 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:14:52.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:13:57.473+0000 7f6778e0e780 0 load: jerasure load: lrc 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:488: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:14:52.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:14:52.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:14:52.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:14:52.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:14:52.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:14:52.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:14:52.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:14:52.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:14:52.665 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:14:52.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:14:53.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:53.674 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:53.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:53.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:53.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:53.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:53.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:14:53.951 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:14:53.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:53.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:53.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:54.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:14:54.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:14:54.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:14:54.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.013 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:54.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607563 2026-02-17T12:14:54.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607563 2026-02-17T12:14:54.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563' 2026-02-17T12:14:54.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.074 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:54.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444042 2026-02-17T12:14:54.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444042 2026-02-17T12:14:54.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-73014444042' 2026-02-17T12:14:54.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:54.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182409 2026-02-17T12:14:54.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182409 2026-02-17T12:14:54.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-73014444042 3-107374182409' 2026-02-17T12:14:54.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:14:54.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888070 2026-02-17T12:14:54.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888070 2026-02-17T12:14:54.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-73014444042 3-107374182409 4-146028888070' 2026-02-17T12:14:54.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.260 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:14:54.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626437 2026-02-17T12:14:54.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626437 2026-02-17T12:14:54.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-73014444042 3-107374182409 4-146028888070 5-180388626437' 2026-02-17T12:14:54.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:54.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364803 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364803 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-73014444042 3-107374182409 4-146028888070 5-180388626437 6-214748364803' 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:14:54.383 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:54.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:14:54.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:14:54.385 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:54.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:14:54.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:14:54.386 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803789 2026-02-17T12:14:54.386 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:54.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:14:54.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:55.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:14:55.578 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:55.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:14:55.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:56.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:14:56.778 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:56.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:14:56.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:56.970 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607563 2026-02-17T12:14:56.970 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:56.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:14:56.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607563 2026-02-17T12:14:56.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:56.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607563 2026-02-17T12:14:56.973 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607563 2026-02-17T12:14:56.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607563' 2026-02-17T12:14:56.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:14:57.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607564 -lt 51539607563 2026-02-17T12:14:57.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:57.159 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444042 2026-02-17T12:14:57.159 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:57.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:14:57.160 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444042 2026-02-17T12:14:57.160 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:57.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444042 2026-02-17T12:14:57.161 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 73014444042 2026-02-17T12:14:57.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-17T12:14:57.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:57.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-17T12:14:57.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:57.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182409 2026-02-17T12:14:57.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:57.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:14:57.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182409 2026-02-17T12:14:57.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:57.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182409 2026-02-17T12:14:57.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-17T12:14:57.355 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 107374182409 2026-02-17T12:14:57.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:14:57.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182410 -lt 107374182409 2026-02-17T12:14:57.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:57.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888070 2026-02-17T12:14:57.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:57.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:14:57.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888070 2026-02-17T12:14:57.544 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:57.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888070 2026-02-17T12:14:57.545 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888070 2026-02-17T12:14:57.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888070' 2026-02-17T12:14:57.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:14:57.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888071 -lt 146028888070 2026-02-17T12:14:57.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:57.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626437 2026-02-17T12:14:57.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:57.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:14:57.739 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626437 2026-02-17T12:14:57.739 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:57.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626437 2026-02-17T12:14:57.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626437' 2026-02-17T12:14:57.741 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626437 2026-02-17T12:14:57.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:14:57.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626437 -lt 180388626437 2026-02-17T12:14:57.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:57.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-214748364803 2026-02-17T12:14:57.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:57.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:14:57.930 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-214748364803 2026-02-17T12:14:57.930 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:57.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364803 2026-02-17T12:14:57.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-17T12:14:57.931 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 214748364803 2026-02-17T12:14:57.931 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:14:58.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-17T12:14:58.119 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:14:58.120 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:58.120 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:58.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:14:58.371 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:14:58.557 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:14:58.557 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:14:58.557 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:58.557 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:58.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:14:58.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:14:58.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:14:58.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:14:58.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:14:58.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:14:58.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-17T12:14:58.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-17T12:14:58.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:14:58.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:14:58.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-17T12:14:58.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:496: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-17T12:14:58.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:14:58.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:14:58.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:14:58.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:14:58.871 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:14:58.872 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:14:59.062 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2026-02-17T12:14:59.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-17T12:14:59.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:14:59.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:14:59.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:14:59.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:14:59.065 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:14:59.066 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:59.066 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:59.066 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:14:59.066 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:14:59.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:14:59.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:14:59.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:14:59.115 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:14:59.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:14:59.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:14:59.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-17T12:14:59.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:497: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-17T12:14:59.405 INFO:tasks.workunit.client.0.trial044.stderr:osd.0 is already out. 2026-02-17T12:14:59.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-17T12:14:59.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:59.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:59.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:59.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:59.416 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:59.416 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:59.416 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:59.416 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:59.416 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:59.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:59.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:59.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:59.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:59.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803792 2026-02-17T12:14:59.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803792 2026-02-17T12:14:59.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792' 2026-02-17T12:14:59.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:59.754 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:59.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607566 2026-02-17T12:14:59.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607566 2026-02-17T12:14:59.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566' 2026-02-17T12:14:59.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:59.817 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:59.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444045 2026-02-17T12:14:59.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444045 2026-02-17T12:14:59.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566 2-73014444045' 2026-02-17T12:14:59.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:59.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:59.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182412 2026-02-17T12:14:59.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182412 2026-02-17T12:14:59.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566 2-73014444045 3-107374182412' 2026-02-17T12:14:59.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:59.940 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:15:00.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888074 2026-02-17T12:15:00.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888074 2026-02-17T12:15:00.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566 2-73014444045 3-107374182412 4-146028888074' 2026-02-17T12:15:00.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:00.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:15:00.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626440 2026-02-17T12:15:00.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626440 2026-02-17T12:15:00.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566 2-73014444045 3-107374182412 4-146028888074 5-180388626440' 2026-02-17T12:15:00.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:00.064 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364806 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364806 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607566 2-73014444045 3-107374182412 4-146028888074 5-180388626440 6-214748364806' 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803792 2026-02-17T12:15:00.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:00.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:00.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803792 2026-02-17T12:15:00.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:00.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803792 2026-02-17T12:15:00.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-17T12:15:00.128 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803792 2026-02-17T12:15:00.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:00.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-17T12:15:00.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:00.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607566 2026-02-17T12:15:00.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:00.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:15:00.319 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607566 2026-02-17T12:15:00.319 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:00.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607566 2026-02-17T12:15:00.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607566' 2026-02-17T12:15:00.321 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607566 2026-02-17T12:15:00.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:15:00.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607566 -lt 51539607566 2026-02-17T12:15:00.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:00.509 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444045 2026-02-17T12:15:00.509 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:00.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:15:00.511 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444045 2026-02-17T12:15:00.511 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:00.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444045 2026-02-17T12:15:00.512 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 73014444045 2026-02-17T12:15:00.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444045' 2026-02-17T12:15:00.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:15:00.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444045 -lt 73014444045 2026-02-17T12:15:00.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:00.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182412 2026-02-17T12:15:00.701 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:00.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:15:00.702 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182412 2026-02-17T12:15:00.702 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:00.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182412 2026-02-17T12:15:00.703 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 107374182412 2026-02-17T12:15:00.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-17T12:15:00.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:00.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182410 -lt 107374182412 2026-02-17T12:15:00.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:01.897 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:01.897 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:02.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182413 -lt 107374182412 2026-02-17T12:15:02.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:02.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888074 2026-02-17T12:15:02.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:02.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:15:02.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888074 2026-02-17T12:15:02.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:02.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888074 2026-02-17T12:15:02.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888074' 2026-02-17T12:15:02.093 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888074 2026-02-17T12:15:02.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:15:02.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888074 -lt 146028888074 2026-02-17T12:15:02.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:02.279 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626440 2026-02-17T12:15:02.279 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:02.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:15:02.280 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626440 2026-02-17T12:15:02.280 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:02.281 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626440 2026-02-17T12:15:02.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626440' 2026-02-17T12:15:02.282 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626440 2026-02-17T12:15:02.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:15:02.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626440 -lt 180388626440 2026-02-17T12:15:02.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:02.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-214748364806 2026-02-17T12:15:02.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:02.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:15:02.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-214748364806 2026-02-17T12:15:02.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:02.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364806 2026-02-17T12:15:02.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 214748364806' 2026-02-17T12:15:02.480 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 214748364806 2026-02-17T12:15:02.480 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:15:02.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364806 -lt 214748364806 2026-02-17T12:15:02.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:02.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:02.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:02.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:02.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:03.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:15:03.115 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:03.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:03.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:03.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:502: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:15:03.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-17T12:15:03.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:15:03.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:15:03.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-17T12:15:03.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:15:03.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:15:03.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-17T12:15:03.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:15:03.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:15:03.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-17T12:15:03.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:15:03.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:15:03.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-17T12:15:03.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:15:03.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:15:03.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:15:03.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:15:03.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:15:03.666 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:15:03.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:15:03.878 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:15:03.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:15:03.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:15:03.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:15:03.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:15:03.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:03.886 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:03.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:03.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:03.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:03.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:03.998 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:15:03.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:15:03.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:15:04.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:15:04.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:15:04.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:15:04.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:04.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:15:04.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:15:04.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:04.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:15:04.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:15:04.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:15:04.020 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:15:04.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:15:04.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:04.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:04.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:04.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:15:04.025 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:15:04.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:15:04.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:15:04.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:15:04.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:15:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:04.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:15:04.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:15:04.028 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:04.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:15:04.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:15:04.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:15:04.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:15:04.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:15:04.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:15:04.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:15:04.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:15:04.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:15:04.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:15:04.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:15:04.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:15:04.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:15:04.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:15:04.055 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:04.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:15:04.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:15:04.082 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:15:04.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:15:04.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:15:04.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:15:04.128 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:15:04.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:15:04.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:15:04.273 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:04.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:04.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:15:04.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:15:04.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:15:04.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:15:04.387 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:15:04.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:15:05.436 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:15:05.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:15:05.442 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:15:04.073+0000 7fb15c5d2d80 0 load: jerasure load: lrc 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:610: TEST_ec_recovery_unfound: local objname=myobject 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_recovery_unfound: local lastobj=100 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:15:05.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:616: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-17T12:15:05.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:05.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_recovery_unfound: setup_osds 5 2026-02-17T12:15:05.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-17T12:15:05.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:15:05.444 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:15:05.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:05.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:05.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=90ec9e73-8701-432f-8ec5-e61d7b3ea995 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 90ec9e73-8701-432f-8ec5-e61d7b3ea995' 2026-02-17T12:15:05.451 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 90ec9e73-8701-432f-8ec5-e61d7b3ea995 2026-02-17T12:15:05.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:05.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDJW5Rp/JyRGxAATpmn8/czJuua+P+Ntbcvwg== 2026-02-17T12:15:05.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDJW5Rp/JyRGxAATpmn8/czJuua+P+Ntbcvwg=="}' 2026-02-17T12:15:05.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 90ec9e73-8701-432f-8ec5-e61d7b3ea995 -i td/test-erasure-eio/0/new.json 2026-02-17T12:15:05.569 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:05.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:15:05.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJW5Rp/JyRGxAATpmn8/czJuua+P+Ntbcvwg== --osd-uuid 90ec9e73-8701-432f-8ec5-e61d7b3ea995 2026-02-17T12:15:05.598 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:05.597+0000 7f85ebc10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:05.599 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:05.598+0000 7f85ebc10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:05.600 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:05.599+0000 7f85ebc10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:05.601 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:05.600+0000 7f85ebc10780 -1 bdev(0x563442889c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:05.601 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:05.600+0000 7f85ebc10780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:15:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:15:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:15:07.971 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:15:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:08.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:15:08.235 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:15:08.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:08.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:08.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:08.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:08.256 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:08.255+0000 7f1786320780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:08.257 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:08.256+0000 7f1786320780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:08.258 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:08.257+0000 7f1786320780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:08.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:15:08.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:08.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:08.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:08.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:09.593 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:09.592+0000 7f1786320780 -1 Falling back to public interface 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:09.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:09.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:10.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:10.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:10.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:10.829 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:10.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:10.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:11.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:12.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:12.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:12.023 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:12.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:12.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:12.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:12.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:12.272 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:12.271+0000 7f1786320780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:15:13.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:13.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:13.215 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:13.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:13.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:13.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:13.409 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2813359119,v1:127.0.0.1:6803/2813359119] [v2:127.0.0.1:6804/2813359119,v1:127.0.0.1:6805/2813359119] exists,up 90ec9e73-8701-432f-8ec5-e61d7b3ea995 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:13.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:13.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:15:13.413 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:13.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef 2026-02-17T12:15:13.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef' 2026-02-17T12:15:13.413 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef 2026-02-17T12:15:13.413 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDRW5RpGGhwGRAAQ41cazklXdPpJv+DuxSQzA== 2026-02-17T12:15:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDRW5RpGGhwGRAAQ41cazklXdPpJv+DuxSQzA=="}' 2026-02-17T12:15:13.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef -i td/test-erasure-eio/1/new.json 2026-02-17T12:15:13.623 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:13.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:15:13.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRW5RpGGhwGRAAQ41cazklXdPpJv+DuxSQzA== --osd-uuid 6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef 2026-02-17T12:15:13.654 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:13.653+0000 7fd94641d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.656 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:13.654+0000 7fd94641d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.657 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:13.656+0000 7fd94641d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.657 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:13.656+0000 7fd94641d780 -1 bdev(0x55b0c89b9c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:13.657 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:13.656+0000 7fd94641d780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:15:15.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:15:15.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:15.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:15:15.784 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:15:15.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:16.063 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:16.062+0000 7f82a4e9b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.065 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:16.064+0000 7f82a4e9b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.066 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:16.065+0000 7f82a4e9b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:15:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:15:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:16.238 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:16.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:16.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:15:16.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:17.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:15:17.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:17.650 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:17.648+0000 7f82a4e9b780 -1 Falling back to public interface 2026-02-17T12:15:18.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:18.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:18.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:18.622 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:18.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:18.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:15:18.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:19.820 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:19.820 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:19.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:19.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:19.821 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:15:20.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:20.583 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:20.581+0000 7f82a4e9b780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:15:21.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:21.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:21.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:21.016 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:21.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:21.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:15:21.211 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1388846727,v1:127.0.0.1:6811/1388846727] [v2:127.0.0.1:6812/1388846727,v1:127.0.0.1:6813/1388846727] exists,up 6a72a0ee-35d3-4da1-b57c-b5b7b193d9ef 2026-02-17T12:15:21.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:15:21.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:21.217 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:21.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5e86c907-0e53-4367-bdfd-a3152ae9bdcc 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 5e86c907-0e53-4367-bdfd-a3152ae9bdcc' 2026-02-17T12:15:21.219 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 5e86c907-0e53-4367-bdfd-a3152ae9bdcc 2026-02-17T12:15:21.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDZW5Rp8seWDRAAtHeVfP6vDRyxC7com3qvqA== 2026-02-17T12:15:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDZW5Rp8seWDRAAtHeVfP6vDRyxC7com3qvqA=="}' 2026-02-17T12:15:21.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5e86c907-0e53-4367-bdfd-a3152ae9bdcc -i td/test-erasure-eio/2/new.json 2026-02-17T12:15:21.426 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:21.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:15:21.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZW5Rp8seWDRAAtHeVfP6vDRyxC7com3qvqA== --osd-uuid 5e86c907-0e53-4367-bdfd-a3152ae9bdcc 2026-02-17T12:15:21.458 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:21.455+0000 7fc00160f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:21.459 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:21.457+0000 7fc00160f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:21.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:21.458+0000 7fc00160f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:21.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:21.458+0000 7fc00160f780 -1 bdev(0x5608adde5c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:21.460 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:21.458+0000 7fc00160f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:15:23.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:15:23.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:23.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:15:23.579 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:15:23.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:23.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:15:23.839 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:15:23.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:23.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:23.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:23.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:23.860 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:23.857+0000 7f940cceb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:23.861 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:23.858+0000 7f940cceb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:23.862 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:23.859+0000 7f940cceb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:24.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:15:24.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:24.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:15:24.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:24.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:24.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:24.040 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:24.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:24.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:24.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:24.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:25.197 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:25.195+0000 7f940cceb780 -1 Falling back to public interface 2026-02-17T12:15:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:25.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:25.233 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:25.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:25.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:25.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:26.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:26.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:26.435 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:26.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:26.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:26.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:26.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:27.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:27.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:27.628 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:27.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:27.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:27.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:27.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:28.156 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:28.154+0000 7f940cceb780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:15:28.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:28.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:28.826 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:28.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:28.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:28.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:29.021 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/780560768,v1:127.0.0.1:6819/780560768] [v2:127.0.0.1:6820/780560768,v1:127.0.0.1:6821/780560768] exists,up 5e86c907-0e53-4367-bdfd-a3152ae9bdcc 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:29.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:29.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:15:29.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:29.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bd10aacb-f28d-4e18-b727-0dc927dcedda 2026-02-17T12:15:29.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 bd10aacb-f28d-4e18-b727-0dc927dcedda' 2026-02-17T12:15:29.024 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 bd10aacb-f28d-4e18-b727-0dc927dcedda 2026-02-17T12:15:29.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:29.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDhW5RpzK9MAhAAqd0UTszO2Wbu97ksb81kJg== 2026-02-17T12:15:29.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDhW5RpzK9MAhAAqd0UTszO2Wbu97ksb81kJg=="}' 2026-02-17T12:15:29.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bd10aacb-f28d-4e18-b727-0dc927dcedda -i td/test-erasure-eio/3/new.json 2026-02-17T12:15:29.239 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:29.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:15:29.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDhW5RpzK9MAhAAqd0UTszO2Wbu97ksb81kJg== --osd-uuid bd10aacb-f28d-4e18-b727-0dc927dcedda 2026-02-17T12:15:29.269 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:29.267+0000 7f4a7940f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:29.270 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:29.269+0000 7f4a7940f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:29.271 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:29.269+0000 7f4a7940f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:29.271 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:29.270+0000 7f4a7940f780 -1 bdev(0x56126a485c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:29.271 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:29.270+0000 7f4a7940f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:15:31.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:15:31.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:31.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:15:31.393 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:15:31.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:31.652 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:15:31.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:15:31.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:31.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:31.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:31.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:31.673 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:31.672+0000 7fa7e041e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:31.674 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:31.673+0000 7fa7e041e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:31.675 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:31.674+0000 7fa7e041e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:31.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:31.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:31.847 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:31.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:31.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:32.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:33.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:33.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:33.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:33.040 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:33.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:33.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:33.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:33.260 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:33.258+0000 7fa7e041e780 -1 Falling back to public interface 2026-02-17T12:15:34.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:34.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:34.238 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:34.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:34.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:34.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:34.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:35.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:35.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:35.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:35.442 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:35.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:35.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:35.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:36.212 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:36.211+0000 7fa7e041e780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:15:36.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:36.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:36.644 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:36.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:36.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:36.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:36.840 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/615372242,v1:127.0.0.1:6827/615372242] [v2:127.0.0.1:6828/615372242,v1:127.0.0.1:6829/615372242] exists,up bd10aacb-f28d-4e18-b727-0dc927dcedda 2026-02-17T12:15:36.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:36.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:15:36.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:36.845 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:15:36.846 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:36.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e88d0acc-7804-48f2-a4ee-773e11a5faea 2026-02-17T12:15:36.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 e88d0acc-7804-48f2-a4ee-773e11a5faea' 2026-02-17T12:15:36.847 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 e88d0acc-7804-48f2-a4ee-773e11a5faea 2026-02-17T12:15:36.847 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:36.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDoW5RpTjMqMxAAOkti3UN7bnpviGTI78Jugw== 2026-02-17T12:15:36.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDoW5RpTjMqMxAAOkti3UN7bnpviGTI78Jugw=="}' 2026-02-17T12:15:36.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e88d0acc-7804-48f2-a4ee-773e11a5faea -i td/test-erasure-eio/4/new.json 2026-02-17T12:15:37.065 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:37.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:15:37.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDoW5RpTjMqMxAAOkti3UN7bnpviGTI78Jugw== --osd-uuid e88d0acc-7804-48f2-a4ee-773e11a5faea 2026-02-17T12:15:37.096 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:37.095+0000 7f1bb381a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:37.097 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:37.096+0000 7f1bb381a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:37.098 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:37.097+0000 7f1bb381a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:37.098 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:37.097+0000 7f1bb381a780 -1 bdev(0x5642e9267c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:37.098 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:37.097+0000 7f1bb381a780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:15:39.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:15:39.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:39.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:15:39.225 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:15:39.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:39.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:15:39.495 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:15:39.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:39.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:39.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:39.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:39.516 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:39.515+0000 7f535f417780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:39.517 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:39.516+0000 7f535f417780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:39.518 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:39.517+0000 7f535f417780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:39.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:15:39.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:40.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:40.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:40.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:40.887 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:40.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:40.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:15:41.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:41.110 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:41.108+0000 7f535f417780 -1 Falling back to public interface 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:42.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:15:42.275 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:43.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:15:43.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:44.076 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:44.075+0000 7f535f417780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:15:44.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:44.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:44.469 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:15:44.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:44.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:44.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:15:44.665 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 31 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/434946622,v1:127.0.0.1:6835/434946622] [v2:127.0.0.1:6836/434946622,v1:127.0.0.1:6837/434946622] exists,up e88d0acc-7804-48f2-a4ee-773e11a5faea 2026-02-17T12:15:44.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:44.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:44.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:44.667 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:15:44.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:15:44.713 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:15:44.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:15:44.720 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:15:09.596+0000 7f1786320780 0 load: jerasure load: lrc 2026-02-17T12:15:44.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:619: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:15:44.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:621: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-17T12:15:44.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:15:44.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:15:44.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:15:44.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:15:44.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:15:44.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:15:44.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:15:44.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:15:45.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:15:45.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:15:45.309 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:15:45.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:15:46.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:15:46.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:15:46.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:15:46.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:15:46.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:15:46.321 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:15:46.321 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:15:46.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:15:46.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:15:46.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:15:46.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:4' 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:46.591 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:15:46.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803786 2026-02-17T12:15:46.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803786 2026-02-17T12:15:46.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786' 2026-02-17T12:15:46.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:46.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:15:46.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607559 2026-02-17T12:15:46.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607559 2026-02-17T12:15:46.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607559' 2026-02-17T12:15:46.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:46.716 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:15:46.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444038 2026-02-17T12:15:46.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444038 2026-02-17T12:15:46.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607559 2-73014444038' 2026-02-17T12:15:46.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:46.777 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:15:46.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215108 2026-02-17T12:15:46.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215108 2026-02-17T12:15:46.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607559 2-73014444038 3-103079215108' 2026-02-17T12:15:46.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:46.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:15:46.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986179 2026-02-17T12:15:46.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986179 2026-02-17T12:15:46.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803786 1-51539607559 2-73014444038 3-103079215108 4-133143986179' 2026-02-17T12:15:46.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:46.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803786 2026-02-17T12:15:46.903 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:46.904 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:46.904 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803786 2026-02-17T12:15:46.905 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:46.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803786 2026-02-17T12:15:46.906 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-17T12:15:46.906 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803786 2026-02-17T12:15:46.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:47.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-17T12:15:47.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:48.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:48.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:48.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-17T12:15:48.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:49.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:15:49.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:49.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-17T12:15:49.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:49.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607559 2026-02-17T12:15:49.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:49.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:15:49.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607559 2026-02-17T12:15:49.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:49.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607559 2026-02-17T12:15:49.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-17T12:15:49.481 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607559 2026-02-17T12:15:49.481 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:15:49.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-17T12:15:49.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:49.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444038 2026-02-17T12:15:49.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:49.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:15:49.674 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444038 2026-02-17T12:15:49.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:49.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444038 2026-02-17T12:15:49.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-17T12:15:49.676 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 73014444038 2026-02-17T12:15:49.676 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:15:49.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-17T12:15:49.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:49.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-103079215108 2026-02-17T12:15:49.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:49.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:15:49.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-103079215108 2026-02-17T12:15:49.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:49.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215108 2026-02-17T12:15:49.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 103079215108' 2026-02-17T12:15:49.868 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 103079215108 2026-02-17T12:15:49.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:50.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215108 2026-02-17T12:15:50.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:50.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-133143986179 2026-02-17T12:15:50.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:50.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:15:50.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-133143986179 2026-02-17T12:15:50.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:50.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986179 2026-02-17T12:15:50.060 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 133143986179 2026-02-17T12:15:50.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 133143986179' 2026-02-17T12:15:50.061 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:15:50.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986179 -lt 133143986179 2026-02-17T12:15:50.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:50.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:50.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:50.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:50.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:50.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:15:50.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:50.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:50.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:50.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:15:50.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:50.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:50.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:46.060062+0000 0'0 39:15 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-17T12:15:45.259194+0000 0'0 2026-02-17T12:15:45.259194+0000 0 0 periodic scrub scheduled @ 2026-02-18T18:30:02.643651+0000 0 0 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.915034+0000 0'0 39:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:55:16.362533+0000 0 0 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.914830+0000 0'0 38:86 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:58:10.325582+0000 0 0 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.853781+0000 0'0 39:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:06:54.642158+0000 0 0 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:28.844705+0000 0'0 38:58 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T22:52:00.269174+0000 0 0 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:15:51.133 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:626: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:15:51.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:51.161 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:628: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-17T12:15:51.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:15:51.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:15:51.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:15:51.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:628: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:628: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local last_osd=2 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:51.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:51.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:52.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:52.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-17T12:15:52.863 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already down. 2026-02-17T12:15:52.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:632: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-17T12:15:53.115 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already out. 2026-02-17T12:15:53.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:634: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:2.0 1 0 1 0 0 4096 0 0 1 0 1 active+undersized+degraded 2026-02-17T12:15:51.552007+0000 39'1 41:25 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-17T12:15:45.259194+0000 0'0 2026-02-17T12:15:45.259194+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.915034+0000 0'0 39:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T19:55:16.362533+0000 0 0 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.914830+0000 0'0 38:86 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:58:10.325582+0000 0 0 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.853781+0000 0'0 39:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:06:54.642158+0000 0 0 2026-02-17T12:15:53.304 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T12:15:51.551950+0000 0'0 41:67 [1,0] 1 [1,0] 1 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:15:53.305 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:15:53.305 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:15:53.305 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:15:53.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-17T12:15:53.312 INFO:tasks.workunit.client.0.trial044.stderr:4+0 records in 2026-02-17T12:15:53.312 INFO:tasks.workunit.client.0.trial044.stderr:4+0 records out 2026-02-17T12:15:53.313 INFO:tasks.workunit.client.0.trial044.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 9.1479e-05 s, 44.8 MB/s 2026-02-17T12:15:53.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: seq 1 100 2026-02-17T12:15:53.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.371 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.392 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.615 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.615 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:53.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:53.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.534 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:54.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:54.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:15:55.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:15:55.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj75 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj75 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj75 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-17T12:15:55.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:15:55.349 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:2147483647' 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2147483647 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:15:55.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:15:55.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:15:55.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:15:55.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:15:55.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:15:55.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:15:55.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:643: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj75 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj75 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:15:55.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj75 2026-02-17T12:15:55.450 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-17T12:15:55.450 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:2147483647' 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 4 0 2147483647 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:15:55.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/1/type 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:15:55.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:55.639 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:15:55.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:15:55.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:15:55.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:55.689 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:55.690 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:55.690 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:15:55.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:15:55.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:645: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:55.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:55.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:15:55.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:15:55.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:15:55.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:15:55.740 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:15:55.755 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:55.754+0000 7f7a2bc1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:55.757 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:55.756+0000 7f7a2bc1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:55.758 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:55.757+0000 7f7a2bc1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:55.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:55.929 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:15:55.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:55.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:56.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:57.113 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:57.111+0000 7f7a2bc1a780 -1 Falling back to public interface 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:57.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:57.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:58.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:58.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:58.308 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:15:58.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:58.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:58.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:58.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:59.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:59.502 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:15:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:59.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:59.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:59.846 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:15:59.843+0000 7f7a2bc1a780 -1 osd.2 39 log_to_monitors true 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:00.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:00.894 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up out weight 0 up_from 45 up_thru 17 down_at 40 last_clean_interval [17,39) [v2:127.0.0.1:6818/1314043764,v1:127.0.0.1:6819/1314043764] [v2:127.0.0.1:6820/1314043764,v1:127.0.0.1:6821/1314043764] exists,up 5e86c907-0e53-4367-bdfd-a3152ae9bdcc 2026-02-17T12:16:00.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:00.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:00.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:00.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-17T12:16:01.141 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already in. 2026-02-17T12:16:01.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: sleep 15 2026-02-17T12:16:16.152 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: seq 1 100 2026-02-17T12:16:16.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-17T12:16:16.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: get_state 2.0 2026-02-17T12:16:16.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-17T12:16:16.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-17T12:16:16.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-17T12:16:16.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-17T12:16:16.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-17T12:16:16.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-17T12:16:16.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-17T12:16:16.345 INFO:tasks.workunit.client.0.trial044.stdout:active+recovery_unfound+degraded 2026-02-17T12:16:16.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:653: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-17T12:16:16.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: break 2026-02-17T12:16:16.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:660: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-17T12:16:16.527 INFO:tasks.workunit.client.0.trial044.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-17T12:16:01.194098+0000 43'101 49:357 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-17T12:15:45.259194+0000 0'0 2026-02-17T12:15:45.259194+0000 0 0 no scrub is scheduled 0 0 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:16:01.192124+0000 0'0 49:55 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T13:19:24.513059+0000 0 0 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.914830+0000 0'0 49:106 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T12:58:10.325582+0000 0 0 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:15:44.853781+0000 0'0 49:40 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:06:54.642158+0000 0 0 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T12:16:01.192115+0000 0'0 49:94 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T12:15:04.368777+0000 0'0 2026-02-17T12:15:04.368777+0000 0 0 periodic scrub scheduled @ 2026-02-18T13:46:30.162963+0000 0 0 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout: 2026-02-17T12:16:16.528 INFO:tasks.workunit.client.0.trial044.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:16:16.529 INFO:tasks.workunit.client.0.trial044.stderr:dumped pgs 2026-02-17T12:16:16.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:661: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout:{ 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: "num_missing": 1, 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: "num_unfound": 1, 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: "objects": [ 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: "oid": { 2026-02-17T12:16:16.596 INFO:tasks.workunit.client.0.trial044.stdout: "oid": "obj75", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "key": "", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "snapid": -2, 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "hash": 3565005477, 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "max": 0, 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "pool": 2, 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "namespace": "" 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "need": "43'76", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "have": "0'0", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "flags": "none", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-17T12:16:16.597 INFO:tasks.workunit.client.0.trial044.stdout: "locations": [ 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: ] 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "state": "NotRecovering", 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "available_might_have_unfound": true, 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "might_have_unfound": [], 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout: "more": false 2026-02-17T12:16:16.598 INFO:tasks.workunit.client.0.trial044.stdout:} 2026-02-17T12:16:16.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-17T12:16:16.662 INFO:tasks.workunit.client.0.trial044.stdout:{ 2026-02-17T12:16:16.662 INFO:tasks.workunit.client.0.trial044.stdout: "snap_trimq": "[]", 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: "snap_trimq_len": 0, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+recovery_unfound+degraded", 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: "epoch": 49, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.663 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "acting_recovery_backfill": [ 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "2(4)", 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:16:16.664 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "info": { 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s0", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "0", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "43'101", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "43'75", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "0'0", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 101, 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:16:16.665 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 37, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 37, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 48, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 37, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 48, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 48, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 37, 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.666 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "version": "43'101", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 358, 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 49, 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+recovery_unfound+degraded", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.667 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:15:51.159420+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:16:01.192325+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:16:01.192325+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:16:01.190525+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:16:01.194098+0000", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 48, 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "0'0", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "0'0", 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "created": 37, 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.668 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 0, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:16:16.669 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:16:16.670 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 413696, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 101, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 505, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 1, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 1, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 3, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 1, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 101, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 101, 2026-02-17T12:16:16.671 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 404, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:16:16.672 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:16:16.673 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.674 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 100 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),4(2)", 2026-02-17T12:16:16.675 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 1 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.676 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "peer_info": [ 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "0(3)", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s3", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "3", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "43'101", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "43'101", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "0'0", 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 101, 2026-02-17T12:16:16.677 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 37, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 37, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 48, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 37, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 48, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 48, 2026-02-17T12:16:16.678 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 37, 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "version": "43'101", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 346, 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 48, 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:16:16.679 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:15:51.159420+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:15:51.550770+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:15:51.550704+0000", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 48, 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "0'0", 2026-02-17T12:16:16.680 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "0'0", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "created": 37, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 0, 2026-02-17T12:16:16.681 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T21:39:47.486169+0000", 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:16:16.682 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 413696, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 101, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 505, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 101, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 101, 2026-02-17T12:16:16.683 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 101, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 404, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:16:16.684 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:16:16.685 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.686 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:16:16.687 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 101 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:16:16.688 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.689 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "1(1)", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s1", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "1", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "43'101", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "43'101", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "0'0", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 101, 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:16:16.690 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 37, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 37, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 48, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 37, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 48, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 48, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 37, 2026-02-17T12:16:16.691 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "version": "43'101", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 130, 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 43, 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.692 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:15:51.159420+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:15:51.550770+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:15:51.550704+0000", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 48, 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "0'0", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "0'0", 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "created": 37, 2026-02-17T12:16:16.693 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 0, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:16:16.694 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T21:39:47.486169+0000", 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 413696, 2026-02-17T12:16:16.695 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 101, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 505, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 1, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 101, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 101, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 101, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 404, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:16:16.696 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:16:16.697 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.698 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.699 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 101 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.700 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "2(4)", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s4", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "4", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "43'101", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "43'75", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "0'0", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 101, 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:16:16.701 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 37, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 37, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 48, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 37, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 48, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 48, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 37, 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.702 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "version": "43'101", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 136, 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 47, 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "state": "remapped+peering", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:16:00.205328+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:16:00.103776+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:16:00.103776+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:16:00.027035+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:15:51.159420+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:16:00.205328+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:16:00.205328+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:16:00.205328+0000", 2026-02-17T12:16:16.703 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 48, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "0'0", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "0'0", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "created": 37, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 0, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:16:16.704 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 413696, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 101, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 505, 2026-02-17T12:16:16.705 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 1, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 101, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 101, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 404, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:16:16.706 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.707 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [], 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [], 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:16:16.708 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "peer": "4(2)", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "pgid": "2.0s2", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "shared": "2", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "last_update": "43'101", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "last_complete": "43'101", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "log_tail": "0'0", 2026-02-17T12:16:16.709 INFO:tasks.workunit.client.0.trial044.stdout: "last_user_version": 101, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill": "MAX", 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [], 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "history": { 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_created": 37, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "epoch_pool_created": 37, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_started": 48, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_interval_clean": 37, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_split": 0, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_marked_full": 0, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "same_up_since": 48, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "same_interval_since": 48, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "same_primary_since": 37, 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.710 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "prior_readable_until_ub": 0 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "stats": { 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "version": "43'101", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "reported_seq": 130, 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "reported_epoch": 43, 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "state": "active+undersized+degraded", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_fresh": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_change": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_active": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_peered": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean": "2026-02-17T12:15:51.159420+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_active": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_became_peered": "2026-02-17T12:15:51.552007+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_unstale": "2026-02-17T12:15:55.158116+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_undegraded": "2026-02-17T12:15:51.550770+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "last_fullsized": "2026-02-17T12:15:51.550704+0000", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "mapping_epoch": 48, 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "log_start": "0'0", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_start": "0'0", 2026-02-17T12:16:16.711 INFO:tasks.workunit.client.0.trial044.stdout: "created": 37, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_clean": 38, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "parent": "0.0", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "parent_split_bits": 0, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub": "0'0", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub": "0'0", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_deep_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_clean_scrub_stamp": "2026-02-17T12:15:45.259194+0000", 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "objects_scrubbed": 0, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "log_size": 101, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "log_dups_size": 0, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "ondisk_log_size": 101, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "dirty_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "omap_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "hitset_bytes_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "pin_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "manifest_stats_invalid": false, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrimq_len": 0, 2026-02-17T12:16:16.712 INFO:tasks.workunit.client.0.trial044.stdout: "last_scrub_duration": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-18T21:39:47.486169+0000", 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_duration": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "objects_trimmed": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "snaptrim_duration": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "stat_sum": { 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes": 413696, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects": 101, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_clones": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_object_copies": 505, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_missing": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_degraded": 101, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_misplaced": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_unfound": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_dirty": 101, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_whiteouts": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_read": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_read_kb": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_write": 101, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_write_kb": 404, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_scrub_errors": 0, 2026-02-17T12:16:16.713 INFO:tasks.workunit.client.0.trial044.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_recovered": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_recovered": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_keys_recovered": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_omap": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_kb": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_kb": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_promote": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_high": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_flush_mode_low": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_some": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_evict_mode_full": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_pinned": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_large_omap_objects": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_manifest": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_bytes": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_omap_keys": 0, 2026-02-17T12:16:16.714 INFO:tasks.workunit.client.0.trial044.stdout: "num_objects_repaired": 0 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "up": [ 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "acting": [ 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 3, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 1, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 4, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 0, 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: 2 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "avail_no_missing": [ 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "3(0)", 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "0(3)", 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "1(1)", 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "4(2)" 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: "object_location_counts": [ 2026-02-17T12:16:16.715 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "objects": 101 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "blocked_by": [], 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "up_primary": 3, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "acting_primary": 3, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "purged_snaps": [] 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "empty": 0, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "dne": 0, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "incomplete": 0, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "last_epoch_started": 49, 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "hit_set_history": { 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "current_last_update": "0'0", 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "history": [] 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_state": [ 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "name": "Started/Primary/Active", 2026-02-17T12:16:16.716 INFO:tasks.workunit.client.0.trial044.stdout: "enter_time": "2026-02-17T12:16:01.190410+0000", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "might_have_unfound": [ 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "0(3)", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "1(1)", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "2(4)", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "osd": "4(2)", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "status": "already probed" 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_progress": { 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "backfill_targets": [], 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "waiting_on_backfill": [], 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "last_backfill_started": "MIN", 2026-02-17T12:16:16.717 INFO:tasks.workunit.client.0.trial044.stdout: "backfill_info": { 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "begin": "MIN", 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "end": "MIN", 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "objects": [] 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "peer_backfill_info": [], 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "backfills_in_flight": [], 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "recovering": [], 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "pg_backend": { 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "recovery_ops": [], 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "read_ops": [] 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: { 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "name": "Started", 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "enter_time": "2026-02-17T12:16:01.090260+0000" 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: } 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: ], 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "scrubber": { 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "active": false, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "must_scrub": false, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "must_deep_scrub": false, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "must_repair": false, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "need_auto": false, 2026-02-17T12:16:16.718 INFO:tasks.workunit.client.0.trial044.stdout: "scrub_reg_stamp": "2026-02-18T21:39:47.486169+0000", 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stdout: "schedule": "no scrub is scheduled" 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stdout: }, 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stdout: "agent_state": {} 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stdout:} 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:664: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:16:16.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:664: TEST_ec_recovery_unfound: grep -q obj75 2026-02-17T12:16:16.732 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:16:16.732 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-17T12:16:16.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: check=true 2026-02-17T12:16:16.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:667: TEST_ec_recovery_unfound: test true == true 2026-02-17T12:16:16.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:669: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-17T12:16:16.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:669: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-17T12:16:16.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:669: TEST_ec_recovery_unfound: check=0 2026-02-17T12:16:16.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: test 0 == 0 2026-02-17T12:16:16.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-17T12:16:21.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: test 124 = 124 2026-02-17T12:16:21.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:676: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-17T12:16:21.930 INFO:tasks.workunit.client.0.trial044.stderr:pg has no unfound objects 2026-02-17T12:16:21.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:678: TEST_ec_recovery_unfound: wait_for_clean 2026-02-17T12:16:21.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:16:21.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:16:21.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:16:21.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:16:21.937 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:16:21.937 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:16:21.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:16:21.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:16:21.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:16:22.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:16:22.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:16:22.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:16:22.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:16:22.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:16:22.024 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:16:22.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:16:22.218 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr:4' 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:22.219 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:16:22.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803795 2026-02-17T12:16:22.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803795 2026-02-17T12:16:22.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803795' 2026-02-17T12:16:22.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:22.283 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:16:22.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607569 2026-02-17T12:16:22.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607569 2026-02-17T12:16:22.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803795 1-51539607569' 2026-02-17T12:16:22.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:22.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:16:22.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528327 2026-02-17T12:16:22.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528327 2026-02-17T12:16:22.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803795 1-51539607569 2-193273528327' 2026-02-17T12:16:22.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:22.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:16:22.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215118 2026-02-17T12:16:22.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215118 2026-02-17T12:16:22.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803795 1-51539607569 2-193273528327 3-103079215118' 2026-02-17T12:16:22.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:22.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:16:22.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986188 2026-02-17T12:16:22.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986188 2026-02-17T12:16:22.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803795 1-51539607569 2-193273528327 3-103079215118 4-133143986188' 2026-02-17T12:16:22.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:22.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803795 2026-02-17T12:16:22.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:22.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:16:22.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803795 2026-02-17T12:16:22.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:22.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803795 2026-02-17T12:16:22.541 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-17T12:16:22.541 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803795 2026-02-17T12:16:22.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:22.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803795 2026-02-17T12:16:22.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:16:23.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:16:23.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:23.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803795 2026-02-17T12:16:23.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:16:24.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:16:24.929 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:25.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803796 -lt 25769803795 2026-02-17T12:16:25.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:25.121 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607569 2026-02-17T12:16:25.122 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:25.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:16:25.123 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607569 2026-02-17T12:16:25.123 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:25.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607569 2026-02-17T12:16:25.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607569' 2026-02-17T12:16:25.124 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607569 2026-02-17T12:16:25.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:16:25.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607569 -lt 51539607569 2026-02-17T12:16:25.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:25.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-193273528327 2026-02-17T12:16:25.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:25.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:16:25.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-193273528327 2026-02-17T12:16:25.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:25.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528327 2026-02-17T12:16:25.317 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 193273528327 2026-02-17T12:16:25.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 193273528327' 2026-02-17T12:16:25.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:16:25.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528327 -lt 193273528327 2026-02-17T12:16:25.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:25.505 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-103079215118 2026-02-17T12:16:25.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:25.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:16:25.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-103079215118 2026-02-17T12:16:25.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:25.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215118 2026-02-17T12:16:25.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 103079215118' 2026-02-17T12:16:25.508 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 103079215118 2026-02-17T12:16:25.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:16:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215118 -lt 103079215118 2026-02-17T12:16:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:25.698 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-133143986188 2026-02-17T12:16:25.698 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:16:25.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-133143986188 2026-02-17T12:16:25.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:25.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986188 2026-02-17T12:16:25.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 133143986188' 2026-02-17T12:16:25.701 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 133143986188 2026-02-17T12:16:25.701 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:16:25.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986188 -lt 133143986188 2026-02-17T12:16:25.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:16:25.891 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:25.891 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:26.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:16:26.150 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:16:26.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:16:26.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:16:26.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:16:26.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:16:26.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:16:26.152 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:16:26.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:16:26.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:16:26.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:26.339 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:26.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:16:26.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:16:26.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:16:26.590 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: seq 1 100 2026-02-17T12:16:26.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-17T12:16:26.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-17T12:16:26.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-17T12:16:26.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-17T12:16:26.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-17T12:16:26.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-17T12:16:26.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-17T12:16:26.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-17T12:16:26.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-17T12:16:26.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-17T12:16:26.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-17T12:16:26.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-17T12:16:26.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-17T12:16:26.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-17T12:16:26.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.847 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-17T12:16:26.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-17T12:16:26.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-17T12:16:26.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-17T12:16:26.903 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-17T12:16:26.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-17T12:16:26.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-17T12:16:26.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-17T12:16:26.975 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-17T12:16:26.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:26.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:26.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-17T12:16:26.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-17T12:16:27.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-17T12:16:27.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-17T12:16:27.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-17T12:16:27.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-17T12:16:27.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-17T12:16:27.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-17T12:16:27.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.140 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-17T12:16:27.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-17T12:16:27.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-17T12:16:27.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-17T12:16:27.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-17T12:16:27.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-17T12:16:27.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-17T12:16:27.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-17T12:16:27.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-17T12:16:27.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-17T12:16:27.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-17T12:16:27.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-17T12:16:27.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-17T12:16:27.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-17T12:16:27.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-17T12:16:27.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-17T12:16:27.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-17T12:16:27.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-17T12:16:27.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-17T12:16:27.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-17T12:16:27.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-17T12:16:27.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.529 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-17T12:16:27.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-17T12:16:27.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-17T12:16:27.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-17T12:16:27.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-17T12:16:27.604 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-17T12:16:27.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-17T12:16:27.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-17T12:16:27.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.677 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.677 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-17T12:16:27.677 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-17T12:16:27.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-17T12:16:27.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-17T12:16:27.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-17T12:16:27.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-17T12:16:27.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-17T12:16:27.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-17T12:16:27.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-17T12:16:27.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-17T12:16:27.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-17T12:16:27.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-17T12:16:27.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-17T12:16:27.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-17T12:16:27.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-17T12:16:27.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.954 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-17T12:16:27.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.972 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-17T12:16:27.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-17T12:16:27.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-17T12:16:27.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:27.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:27.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-17T12:16:27.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-17T12:16:28.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-17T12:16:28.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-17T12:16:28.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-17T12:16:28.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-17T12:16:28.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-17T12:16:28.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-17T12:16:28.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-17T12:16:28.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-17T12:16:28.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-17T12:16:28.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-17T12:16:28.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-17T12:16:28.214 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-17T12:16:28.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-17T12:16:28.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-17T12:16:28.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-17T12:16:28.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-17T12:16:28.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-17T12:16:28.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-17T12:16:28.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.360 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-17T12:16:28.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-17T12:16:28.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-17T12:16:28.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-17T12:16:28.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-17T12:16:28.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-17T12:16:28.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:691: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-17T12:16:28.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:16:28.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:16:28.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:16:28.670 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:16:28.677 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:16:28.880 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:16:28.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:16:28.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:16:28.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:16:28.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:16:28.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:16:28.999 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:16:29.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:16:29.000 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:16:29.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:16:29.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:16:29.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:16:29.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:29.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:16:29.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:16:29.003 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:29.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:16:29.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:16:29.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:16:29.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:16:29.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:16:29.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:16:29.030 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:16:29.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:16:29.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:16:29.031 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:16:29.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:16:29.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:16:29.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:16:29.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:29.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:16:29.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:16:29.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:29.035 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:16:29.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:16:29.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:16:29.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:16:29.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:16:29.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:16:29.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:16:29.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:16:29.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:16:29.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:16:29.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:16:29.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:16:29.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:16:29.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:16:29.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:16:29.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:16:29.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:29.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:29.063 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:29.064 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.064 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.064 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:29.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:16:29.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:16:29.091 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:16:29.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:16:29.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:16:29.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:16:29.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:16:29.093 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:16:29.093 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:16:29.093 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:16:29.093 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:16:29.094 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.094 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.094 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:16:29.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:16:29.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:16:29.137 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:16:29.137 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:16:29.138 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:16:29.139 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:16:29.139 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.139 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.139 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:16:29.140 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:16:29.140 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:16:29.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:16:29.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:16:29.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:16:29.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:29.288 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:16:29.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:16:29.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:16:29.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:16:29.402 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:16:29.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:16:30.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:16:30.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:16:30.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:16:30.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:16:30.453 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:16:30.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:16:30.460 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:16:29.082+0000 7fcb6ce3fd80 0 load: jerasure load: lrc 2026-02-17T12:16:30.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:390: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:391: TEST_ec_single_recovery_error: local objname=myobject 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:393: TEST_ec_single_recovery_error: setup_osds 7 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-17T12:16:30.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-17T12:16:30.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:16:30.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:16:30.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:16:30.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:30.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:30.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=cf47dcdb-dbe3-4084-a074-f279d823ae95 2026-02-17T12:16:30.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 cf47dcdb-dbe3-4084-a074-f279d823ae95' 2026-02-17T12:16:30.469 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 cf47dcdb-dbe3-4084-a074-f279d823ae95 2026-02-17T12:16:30.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:30.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAeXJRpo4drHBAAPsw7TX+uddZ3ogj0yKNigQ== 2026-02-17T12:16:30.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAeXJRpo4drHBAAPsw7TX+uddZ3ogj0yKNigQ=="}' 2026-02-17T12:16:30.479 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new cf47dcdb-dbe3-4084-a074-f279d823ae95 -i td/test-erasure-eio/0/new.json 2026-02-17T12:16:30.580 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:16:30.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:16:30.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeXJRpo4drHBAAPsw7TX+uddZ3ogj0yKNigQ== --osd-uuid cf47dcdb-dbe3-4084-a074-f279d823ae95 2026-02-17T12:16:30.606 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:30.604+0000 7ff4aa0e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:30.607 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:30.606+0000 7ff4aa0e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:30.608 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:30.607+0000 7ff4aa0e7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:30.608 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:30.607+0000 7ff4aa0e7780 -1 bdev(0x556cc987fc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:30.609 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:30.607+0000 7ff4aa0e7780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:16:32.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:16:32.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:32.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:16:32.980 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:16:32.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:33.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:16:33.238 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:16:33.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:33.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:33.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:33.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:33.255 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:33.254+0000 7f3d7a61e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:33.256 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:33.255+0000 7f3d7a61e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:33.257 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:33.256+0000 7f3d7a61e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:33.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:33.431 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:16:33.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:33.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:33.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:34.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:34.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:34.847 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:34.845+0000 7f3d7a61e780 -1 Falling back to public interface 2026-02-17T12:16:35.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:35.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:35.803 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:16:35.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:35.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:35.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:35.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:36.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:36.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:36.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:36.998 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:16:36.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:36.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:37.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:37.536 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:37.535+0000 7f3d7a61e780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:16:38.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:38.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:38.189 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:16:38.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:38.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:38.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:38.378 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/786056903,v1:127.0.0.1:6803/786056903] [v2:127.0.0.1:6804/786056903,v1:127.0.0.1:6805/786056903] exists,up cf47dcdb-dbe3-4084-a074-f279d823ae95 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:38.379 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:38.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:16:38.381 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:38.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3c1ba392-2f61-479e-9f93-e7044daf9a8f 2026-02-17T12:16:38.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3c1ba392-2f61-479e-9f93-e7044daf9a8f' 2026-02-17T12:16:38.382 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 3c1ba392-2f61-479e-9f93-e7044daf9a8f 2026-02-17T12:16:38.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:38.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAmXJRpsohvFxAALwMzqeR/npit05O57w135g== 2026-02-17T12:16:38.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAmXJRpsohvFxAALwMzqeR/npit05O57w135g=="}' 2026-02-17T12:16:38.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3c1ba392-2f61-479e-9f93-e7044daf9a8f -i td/test-erasure-eio/1/new.json 2026-02-17T12:16:38.593 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:16:38.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:16:38.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmXJRpsohvFxAALwMzqeR/npit05O57w135g== --osd-uuid 3c1ba392-2f61-479e-9f93-e7044daf9a8f 2026-02-17T12:16:38.619 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:38.617+0000 7f5480e82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:38.620 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:38.619+0000 7f5480e82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:38.621 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:38.620+0000 7f5480e82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:38.621 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:38.620+0000 7f5480e82780 -1 bdev(0x5650c6251c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:38.621 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:38.620+0000 7f5480e82780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:16:40.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:16:40.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:40.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:16:40.748 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:16:40.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:41.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:16:41.020 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:16:41.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:41.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:41.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:41.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:41.036 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:41.034+0000 7fab14562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:41.037 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:41.035+0000 7fab14562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:41.039 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:41.036+0000 7fab14562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:41.207 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:16:41.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:41.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:41.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:42.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:42.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:42.402 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:16:42.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:42.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:42.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:42.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:42.618 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:42.616+0000 7fab14562780 -1 Falling back to public interface 2026-02-17T12:16:43.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:43.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:43.589 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:16:43.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:43.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:43.589 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:43.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:44.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:44.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:45.557 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:45.555+0000 7fab14562780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:45.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:46.162 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/79072775,v1:127.0.0.1:6811/79072775] [v2:127.0.0.1:6812/79072775,v1:127.0.0.1:6813/79072775] exists,up 3c1ba392-2f61-479e-9f93-e7044daf9a8f 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:46.163 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:46.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:16:46.165 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:46.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=313e611b-a426-4a6b-bc7f-f34a85ada617 2026-02-17T12:16:46.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 313e611b-a426-4a6b-bc7f-f34a85ada617' 2026-02-17T12:16:46.166 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 313e611b-a426-4a6b-bc7f-f34a85ada617 2026-02-17T12:16:46.166 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:46.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAuXJRpNhWQChAA5GT2j/mJ/NR0l6lRn2QqOg== 2026-02-17T12:16:46.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAuXJRpNhWQChAA5GT2j/mJ/NR0l6lRn2QqOg=="}' 2026-02-17T12:16:46.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 313e611b-a426-4a6b-bc7f-f34a85ada617 -i td/test-erasure-eio/2/new.json 2026-02-17T12:16:46.376 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:16:46.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:16:46.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAuXJRpNhWQChAA5GT2j/mJ/NR0l6lRn2QqOg== --osd-uuid 313e611b-a426-4a6b-bc7f-f34a85ada617 2026-02-17T12:16:46.403 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:46.401+0000 7f6c49218780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.404 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:46.402+0000 7f6c49218780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.405 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:46.403+0000 7f6c49218780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.405 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:46.403+0000 7f6c49218780 -1 bdev(0x556654d41c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:46.405 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:46.403+0000 7f6c49218780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:16:48.524 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:16:48.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:48.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:16:48.526 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:16:48.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:48.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:16:48.784 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:16:48.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:48.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:48.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:48.785 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:48.801 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:48.800+0000 7f47f3164780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:48.802 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:48.801+0000 7f47f3164780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:48.803 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:48.802+0000 7f47f3164780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:48.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:49.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:50.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:50.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:50.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:50.163 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:16:50.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:50.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:50.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:50.388 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:50.386+0000 7f47f3164780 -1 Falling back to public interface 2026-02-17T12:16:51.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:51.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:51.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:51.355 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:16:51.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:51.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:51.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:52.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:52.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:52.546 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:16:52.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:52.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:52.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:52.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:53.336 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:53.335+0000 7f47f3164780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:16:53.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:53.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:53.738 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:16:53.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:53.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:53.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:53.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:54.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:54.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:54.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:16:54.934 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:16:54.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:54.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:16:55.122 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1513030115,v1:127.0.0.1:6819/1513030115] [v2:127.0.0.1:6820/1513030115,v1:127.0.0.1:6821/1513030115] exists,up 313e611b-a426-4a6b-bc7f-f34a85ada617 2026-02-17T12:16:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:55.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:55.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:55.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:16:55.131 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:55.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d5ba8d37-1900-4421-bf38-50ac7f7539b3 2026-02-17T12:16:55.131 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 d5ba8d37-1900-4421-bf38-50ac7f7539b3 2026-02-17T12:16:55.132 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 d5ba8d37-1900-4421-bf38-50ac7f7539b3' 2026-02-17T12:16:55.132 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:55.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA3XJRp3IMxCBAAG1UMhy/IwfX9y1XIoY8+qw== 2026-02-17T12:16:55.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA3XJRp3IMxCBAAG1UMhy/IwfX9y1XIoY8+qw=="}' 2026-02-17T12:16:55.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d5ba8d37-1900-4421-bf38-50ac7f7539b3 -i td/test-erasure-eio/3/new.json 2026-02-17T12:16:55.332 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:16:55.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:16:55.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3XJRp3IMxCBAAG1UMhy/IwfX9y1XIoY8+qw== --osd-uuid d5ba8d37-1900-4421-bf38-50ac7f7539b3 2026-02-17T12:16:55.358 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:55.357+0000 7fca6e33b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:55.359 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:55.358+0000 7fca6e33b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:55.360 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:55.359+0000 7fca6e33b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:55.361 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:55.360+0000 7fca6e33b780 -1 bdev(0x55b352527c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:55.361 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:55.360+0000 7fca6e33b780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:16:57.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:16:57.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:57.475 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:16:57.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:16:57.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:57.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:16:57.732 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:16:57.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:57.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:57.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:57.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:57.749 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:57.748+0000 7fd51b023780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:57.750 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:57.749+0000 7fd51b023780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:57.751 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:57.750+0000 7fd51b023780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:57.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:57.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:16:58.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:59.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:16:59.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:59.344 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:16:59.342+0000 7fd51b023780 -1 Falling back to public interface 2026-02-17T12:17:00.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:00.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:00.303 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:00.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:00.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:00.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:17:00.490 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:01.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:01.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:01.492 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:01.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:01.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:01.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:17:01.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:02.284 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:02.283+0000 7fd51b023780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:17:02.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:02.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:02.683 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:02.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:02.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:02.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:17:02.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:03.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:17:03.878 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:03.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:17:04.070 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1986944254,v1:127.0.0.1:6827/1986944254] [v2:127.0.0.1:6828/1986944254,v1:127.0.0.1:6829/1986944254] exists,up d5ba8d37-1900-4421-bf38-50ac7f7539b3 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:17:04.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:04.074 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-17T12:17:04.075 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:17:04.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6e24ad22-9867-4f30-9c1f-e6e437210f21 2026-02-17T12:17:04.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 6e24ad22-9867-4f30-9c1f-e6e437210f21' 2026-02-17T12:17:04.076 INFO:tasks.workunit.client.0.trial044.stdout:add osd4 6e24ad22-9867-4f30-9c1f-e6e437210f21 2026-02-17T12:17:04.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:17:04.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBAXJRpuwoEBRAAo9e66+BVxgEelJwoiWBuoA== 2026-02-17T12:17:04.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBAXJRpuwoEBRAAo9e66+BVxgEelJwoiWBuoA=="}' 2026-02-17T12:17:04.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6e24ad22-9867-4f30-9c1f-e6e437210f21 -i td/test-erasure-eio/4/new.json 2026-02-17T12:17:04.280 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:04.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-17T12:17:04.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAXJRpuwoEBRAAo9e66+BVxgEelJwoiWBuoA== --osd-uuid 6e24ad22-9867-4f30-9c1f-e6e437210f21 2026-02-17T12:17:04.306 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:04.305+0000 7f1ca2f37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:04.308 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:04.306+0000 7f1ca2f37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:04.309 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:04.308+0000 7f1ca2f37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:04.309 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:04.308+0000 7f1ca2f37780 -1 bdev(0x563b1bfd7c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-17T12:17:04.309 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:04.308+0000 7f1ca2f37780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-17T12:17:06.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-17T12:17:06.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:17:06.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T12:17:06.428 INFO:tasks.workunit.client.0.trial044.stdout:adding osd4 key to auth repository 2026-02-17T12:17:06.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:17:06.686 INFO:tasks.workunit.client.0.trial044.stdout:start osd.4 2026-02-17T12:17:06.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T12:17:06.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:06.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:17:06.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:17:06.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:17:06.704 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:06.703+0000 7f62fb2d7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:06.706 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:06.705+0000 7f62fb2d7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:06.707 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:06.706+0000 7f62fb2d7780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:06.877 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:06.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:06.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:07.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:08.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:08.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:08.071 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:08.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:08.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:08.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:08.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:08.297 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:08.295+0000 7f62fb2d7780 -1 Falling back to public interface 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:09.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:09.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:10.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:10.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:11.514 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:11.513+0000 7f62fb2d7780 -1 osd.4 0 log_to_monitors true 2026-02-17T12:17:11.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:11.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:11.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:11.642 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:11.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:11.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:11.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:12.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:12.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:12.863 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:12.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:17:12.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:12.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T12:17:13.054 INFO:tasks.workunit.client.0.trial044.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3272130121,v1:127.0.0.1:6835/3272130121] [v2:127.0.0.1:6836/3272130121,v1:127.0.0.1:6837/3272130121] exists,up 6e24ad22-9867-4f30-9c1f-e6e437210f21 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T12:17:13.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:17:13.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:13.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c40c2692-8179-4a93-bb61-25be45c4b4a5 2026-02-17T12:17:13.061 INFO:tasks.workunit.client.0.trial044.stdout:add osd5 c40c2692-8179-4a93-bb61-25be45c4b4a5 2026-02-17T12:17:13.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 c40c2692-8179-4a93-bb61-25be45c4b4a5' 2026-02-17T12:17:13.062 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:17:13.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBJXJRpPF4TBBAADSZDclzdNrGi0CBFe9j7qg== 2026-02-17T12:17:13.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBJXJRpPF4TBBAADSZDclzdNrGi0CBFe9j7qg=="}' 2026-02-17T12:17:13.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c40c2692-8179-4a93-bb61-25be45c4b4a5 -i td/test-erasure-eio/5/new.json 2026-02-17T12:17:13.262 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:13.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-17T12:17:13.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBJXJRpPF4TBBAADSZDclzdNrGi0CBFe9j7qg== --osd-uuid c40c2692-8179-4a93-bb61-25be45c4b4a5 2026-02-17T12:17:13.289 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:13.287+0000 7f4f3c0ce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:13.290 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:13.289+0000 7f4f3c0ce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:13.291 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:13.290+0000 7f4f3c0ce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:13.292 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:13.290+0000 7f4f3c0ce780 -1 bdev(0x55fde2025c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-17T12:17:13.292 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:13.290+0000 7f4f3c0ce780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-17T12:17:15.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-17T12:17:15.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:17:15.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T12:17:15.427 INFO:tasks.workunit.client.0.trial044.stdout:adding osd5 key to auth repository 2026-02-17T12:17:15.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:17:15.689 INFO:tasks.workunit.client.0.trial044.stdout:start osd.5 2026-02-17T12:17:15.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T12:17:15.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:15.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:17:15.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:17:15.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:17:15.706 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:15.705+0000 7f1f8823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:15.707 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:15.706+0000 7f1f8823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:15.708 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:15.707+0000 7f1f8823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:15.881 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:15.882 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:15.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:15.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:16.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:17.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:17.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:17.071 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:17.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:17.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:17.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:17.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:17.289 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:17.287+0000 7f1f8823b780 -1 Falling back to public interface 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:18.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:18.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:19.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:19.456 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:19.457 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:19.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:19.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:19.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:19.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:20.232 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:20.230+0000 7f1f8823b780 -1 osd.5 0 log_to_monitors true 2026-02-17T12:17:20.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:20.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:20.646 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:20.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:20.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:20.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:20.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:21.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:21.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:21.842 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:21.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:17:21.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:21.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3829349152,v1:127.0.0.1:6843/3829349152] [v2:127.0.0.1:6844/3829349152,v1:127.0.0.1:6845/3829349152] exists,up c40c2692-8179-4a93-bb61-25be45c4b4a5 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-17T12:17:22.031 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:22.032 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:17:22.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-17T12:17:22.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:17:22.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6 2026-02-17T12:17:22.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6' 2026-02-17T12:17:22.034 INFO:tasks.workunit.client.0.trial044.stdout:add osd6 4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6 2026-02-17T12:17:22.035 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:17:22.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBSXJRpYDy+AhAA5vWa7FRj78yRkTI5CEsw5Q== 2026-02-17T12:17:22.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBSXJRpYDy+AhAA5vWa7FRj78yRkTI5CEsw5Q=="}' 2026-02-17T12:17:22.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6 -i td/test-erasure-eio/6/new.json 2026-02-17T12:17:22.240 INFO:tasks.workunit.client.0.trial044.stdout:6 2026-02-17T12:17:22.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-17T12:17:22.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBSXJRpYDy+AhAA5vWa7FRj78yRkTI5CEsw5Q== --osd-uuid 4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6 2026-02-17T12:17:22.266 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:22.264+0000 7ff5aac13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:22.267 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:22.265+0000 7ff5aac13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:22.268 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:22.266+0000 7ff5aac13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:22.269 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:22.267+0000 7ff5aac13780 -1 bdev(0x5567b868bc00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-17T12:17:22.269 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:22.267+0000 7ff5aac13780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-17T12:17:24.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-17T12:17:24.387 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:17:24.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T12:17:24.388 INFO:tasks.workunit.client.0.trial044.stdout:adding osd6 key to auth repository 2026-02-17T12:17:24.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:17:24.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T12:17:24.644 INFO:tasks.workunit.client.0.trial044.stdout:start osd.6 2026-02-17T12:17:24.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:24.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:17:24.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:17:24.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:17:24.660 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:24.658+0000 7f4c5c562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:24.661 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:24.659+0000 7f4c5c562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:24.662 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:24.660+0000 7f4c5c562780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:24.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T12:17:24.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:24.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T12:17:24.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:24.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:24.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:24.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:24.832 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:24.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:24.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:25.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:26.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:26.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:26.021 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:26.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:26.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:26.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:26.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:26.257 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:26.254+0000 7f4c5c562780 -1 Falling back to public interface 2026-02-17T12:17:27.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:27.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:27.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:27.210 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:27.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:27.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:27.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:28.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:28.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:28.400 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:28.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:28.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:28.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:28.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:29.210 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:29.209+0000 7f4c5c562780 -1 osd.6 0 log_to_monitors true 2026-02-17T12:17:29.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:29.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:29.592 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:29.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:29.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:29.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:29.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:30.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:30.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:30.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:17:30.791 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:30.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:30.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T12:17:30.981 INFO:tasks.workunit.client.0.trial044.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3870565872,v1:127.0.0.1:6851/3870565872] [v2:127.0.0.1:6852/3870565872,v1:127.0.0.1:6853/3870565872] exists,up 4d3b67b1-d73b-4248-b2a2-31cf68eb6fa6 2026-02-17T12:17:30.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:30.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:30.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:30.983 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:17:30.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:17:31.028 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:17:31.032 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:17:31.035 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:16:34.848+0000 7f3d7a61e780 0 load: jerasure load: lrc 2026-02-17T12:17:31.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:396: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:17:31.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-17T12:17:31.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:17:31.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:17:31.583 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:17:31.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:17:32.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:17:32.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:17:32.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:17:32.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:17:32.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:17:32.593 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:17:32.593 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:17:32.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:17:32.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:17:32.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:17:32.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:17:32.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:17:32.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:17:32.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:17:32.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:17:32.681 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:17:32.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:32.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:17:32.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:17:32.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:17:32.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:17:32.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:32.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:17:32.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607565 2026-02-17T12:17:32.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607565 2026-02-17T12:17:32.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565' 2026-02-17T12:17:32.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:32.991 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:17:33.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411339 2026-02-17T12:17:33.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411339 2026-02-17T12:17:33.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565 2-77309411339' 2026-02-17T12:17:33.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:33.054 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:17:33.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182409 2026-02-17T12:17:33.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182409 2026-02-17T12:17:33.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565 2-77309411339 3-107374182409' 2026-02-17T12:17:33.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:33.115 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:17:33.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888071 2026-02-17T12:17:33.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888071 2026-02-17T12:17:33.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565 2-77309411339 3-107374182409 4-146028888071' 2026-02-17T12:17:33.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:33.180 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:17:33.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626438 2026-02-17T12:17:33.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626438 2026-02-17T12:17:33.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565 2-77309411339 3-107374182409 4-146028888071 5-180388626438' 2026-02-17T12:17:33.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:33.241 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364803 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364803 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607565 2-77309411339 3-107374182409 4-146028888071 5-180388626438 6-214748364803' 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:17:33.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:33.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:17:33.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:17:33.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:33.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:17:33.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:17:33.304 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803789 2026-02-17T12:17:33.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:33.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:17:33.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:33.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607565 2026-02-17T12:17:33.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:33.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:17:33.495 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607565 2026-02-17T12:17:33.495 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:33.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607565 2026-02-17T12:17:33.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607565' 2026-02-17T12:17:33.496 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607565 2026-02-17T12:17:33.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:17:33.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607565 -lt 51539607565 2026-02-17T12:17:33.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:33.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411339 2026-02-17T12:17:33.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:33.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:17:33.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411339 2026-02-17T12:17:33.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:33.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411339 2026-02-17T12:17:33.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411339' 2026-02-17T12:17:33.688 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411339 2026-02-17T12:17:33.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:17:33.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411339 -lt 77309411339 2026-02-17T12:17:33.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:33.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182409 2026-02-17T12:17:33.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:33.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:17:33.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182409 2026-02-17T12:17:33.878 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:33.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182409 2026-02-17T12:17:33.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-17T12:17:33.879 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 107374182409 2026-02-17T12:17:33.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:17:34.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-17T12:17:34.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:34.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888071 2026-02-17T12:17:34.065 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:34.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:17:34.066 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888071 2026-02-17T12:17:34.067 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:34.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888071 2026-02-17T12:17:34.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-17T12:17:34.068 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888071 2026-02-17T12:17:34.068 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:17:34.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-17T12:17:34.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:34.255 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626438 2026-02-17T12:17:34.255 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:34.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:17:34.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626438 2026-02-17T12:17:34.257 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:34.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626438 2026-02-17T12:17:34.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626438' 2026-02-17T12:17:34.258 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626438 2026-02-17T12:17:34.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:17:34.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626438 -lt 180388626438 2026-02-17T12:17:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:34.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-214748364803 2026-02-17T12:17:34.449 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:34.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T12:17:34.450 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-214748364803 2026-02-17T12:17:34.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:34.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364803 2026-02-17T12:17:34.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-17T12:17:34.452 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.6 seq 214748364803 2026-02-17T12:17:34.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T12:17:34.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-17T12:17:34.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:17:34.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:34.642 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:34.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:17:34.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:17:34.894 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:17:35.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:17:35.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:17:35.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:35.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:17:35.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:17:35.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-17T12:17:35.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:399: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-17T12:17:35.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=myobject 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure myobject 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:17:35.353 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:17:35.354 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:17:35.542 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:17:35.542 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:17:35.542 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:17:35.542 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:17:35.543 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:17:35.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:17:35.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:17:35.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:17:35.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:17:35.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:17:35.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:17:35.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:17:35.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-17T12:17:35.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-17T12:17:35.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:17:35.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=myobject 2026-02-17T12:17:35.646 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-17T12:17:35.646 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:17:35.834 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 1 0 6 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:402: TEST_ec_single_recovery_error: local last_osd=6 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:404: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:17:35.835 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:17:35.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:17:35.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:17:35.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:17:37.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:17:37.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-17T12:17:37.335 INFO:tasks.workunit.client.0.trial044.stderr:osd.6 is already down. 2026-02-17T12:17:37.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:406: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-17T12:17:37.589 INFO:tasks.workunit.client.0.trial044.stderr:osd.6 is already out. 2026-02-17T12:17:37.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:409: TEST_ec_single_recovery_error: wait_for_clean 2026-02-17T12:17:37.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:17:37.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:17:37.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:17:37.598 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:17:37.598 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:17:37.599 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:17:37.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:17:37.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:17:37.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:17:37.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:17:37.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:5 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:6' 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:37.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:17:37.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803792 2026-02-17T12:17:37.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803792 2026-02-17T12:17:37.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792' 2026-02-17T12:17:37.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:37.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:17:37.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607568 2026-02-17T12:17:38.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607568 2026-02-17T12:17:38.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607568' 2026-02-17T12:17:38.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:38.000 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:17:38.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411342 2026-02-17T12:17:38.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411342 2026-02-17T12:17:38.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607568 2-77309411342' 2026-02-17T12:17:38.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:38.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:17:38.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182412 2026-02-17T12:17:38.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182412 2026-02-17T12:17:38.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607568 2-77309411342 3-107374182412' 2026-02-17T12:17:38.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:38.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T12:17:38.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888074 2026-02-17T12:17:38.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888074 2026-02-17T12:17:38.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607568 2-77309411342 3-107374182412 4-146028888074' 2026-02-17T12:17:38.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:38.186 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T12:17:38.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626441 2026-02-17T12:17:38.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626441 2026-02-17T12:17:38.248 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803792 1-51539607568 2-77309411342 3-107374182412 4-146028888074 5-180388626441' 2026-02-17T12:17:38.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:38.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T12:17:38.292 INFO:tasks.workunit.client.0.trial044.stderr:Error ENXIO: problem getting command descriptions from osd.6 2026-02-17T12:17:38.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T12:17:38.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T12:17:38.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T12:17:38.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:38.293 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803792 2026-02-17T12:17:38.294 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:38.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:17:38.296 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803792 2026-02-17T12:17:38.296 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:38.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803792 2026-02-17T12:17:38.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-17T12:17:38.297 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803792 2026-02-17T12:17:38.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:38.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-17T12:17:38.493 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:17:39.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:17:39.495 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:39.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803792 2026-02-17T12:17:39.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:39.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607568 2026-02-17T12:17:39.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:39.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:17:39.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607568 2026-02-17T12:17:39.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:39.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607568 2026-02-17T12:17:39.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607568' 2026-02-17T12:17:39.694 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607568 2026-02-17T12:17:39.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:17:39.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607568 -lt 51539607568 2026-02-17T12:17:39.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:39.882 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411342 2026-02-17T12:17:39.882 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:39.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:17:39.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411342 2026-02-17T12:17:39.884 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:39.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411342 2026-02-17T12:17:39.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-17T12:17:39.885 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411342 2026-02-17T12:17:39.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:17:40.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411342 -lt 77309411342 2026-02-17T12:17:40.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:40.075 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182412 2026-02-17T12:17:40.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:17:40.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182412 2026-02-17T12:17:40.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:40.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182412 2026-02-17T12:17:40.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-17T12:17:40.078 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 107374182412 2026-02-17T12:17:40.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:17:40.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182413 -lt 107374182412 2026-02-17T12:17:40.265 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:40.266 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-146028888074 2026-02-17T12:17:40.266 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:40.267 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T12:17:40.267 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-146028888074 2026-02-17T12:17:40.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:40.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888074 2026-02-17T12:17:40.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 146028888074' 2026-02-17T12:17:40.269 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.4 seq 146028888074 2026-02-17T12:17:40.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T12:17:40.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888074 -lt 146028888074 2026-02-17T12:17:40.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:40.463 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-180388626441 2026-02-17T12:17:40.463 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:40.464 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T12:17:40.465 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-180388626441 2026-02-17T12:17:40.465 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:40.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626441 2026-02-17T12:17:40.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 180388626441' 2026-02-17T12:17:40.466 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.5 seq 180388626441 2026-02-17T12:17:40.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T12:17:40.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626441 -lt 180388626441 2026-02-17T12:17:40.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:17:40.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:40.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:40.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:17:40.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:17:40.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:17:41.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:17:41.098 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:17:41.098 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:41.098 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:41.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:17:41.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:17:41.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:17:41.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-17T12:17:41.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:17:41.368 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:17:41.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:17:41.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:17:41.369 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:17:41.562 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:17:41.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:17:41.770 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:17:41.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:17:41.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:17:41.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:17:41.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:17:41.779 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:17:41.779 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:17:41.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:17:41.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:17:41.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:17:41.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:17:41.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:17:41.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:17:41.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:17:41.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:17:41.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:17:41.891 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:17:41.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:17:41.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:17:41.892 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:17:41.893 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:17:41.893 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:17:41.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:17:41.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:17:41.909 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:17:41.910 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:41.910 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:41.910 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:17:41.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:17:41.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:17:41.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:17:41.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:17:41.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:17:41.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:17:41.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:17:41.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:17:41.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:17:41.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:17:41.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:17:41.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:17:41.916 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:17:41.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:17:41.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:17:41.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:17:41.917 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:17:41.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:17:41.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:17:41.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:17:41.920 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:17:41.920 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:41.920 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:41.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:17:41.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:17:41.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:17:41.921 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:17:41.922 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:17:41.922 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:41.922 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:41.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:17:41.923 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:17:41.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:17:41.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:17:41.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:41.946 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:41.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:17:41.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:17:41.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:17:41.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:17:41.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:17:41.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:17:41.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:41.975 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:41.976 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:17:41.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:17:41.976 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:17:42.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:17:42.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:17:42.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:17:42.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:17:42.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:17:42.020 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:17:42.020 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:17:42.020 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:17:42.020 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:42.020 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:42.021 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:42.021 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:17:42.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:17:42.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:17:42.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:17:42.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:17:42.169 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:17:42.169 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:42.169 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:42.170 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:42.170 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:42.170 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:42.170 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:42.170 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:17:42.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:17:42.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:17:42.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:17:42.282 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:17:42.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:43.290 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:43.291 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:17:43.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:17:43.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:17:43.332 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:17:43.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:17:43.338 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:17:41.964+0000 7f202a62cd80 0 load: jerasure load: lrc 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:17:43.339 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:17:43.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:17:43.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:43.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:17:43.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5b4ac019-50d1-4af2-8d30-7c88089dfed7 2026-02-17T12:17:43.347 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 5b4ac019-50d1-4af2-8d30-7c88089dfed7 2026-02-17T12:17:43.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 5b4ac019-50d1-4af2-8d30-7c88089dfed7' 2026-02-17T12:17:43.347 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:17:43.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBnXJRpLKklFRAA7xmAOk0M5jIR0t48UYAp2w== 2026-02-17T12:17:43.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBnXJRpLKklFRAA7xmAOk0M5jIR0t48UYAp2w=="}' 2026-02-17T12:17:43.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5b4ac019-50d1-4af2-8d30-7c88089dfed7 -i td/test-erasure-eio/0/new.json 2026-02-17T12:17:43.459 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:43.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:17:43.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnXJRpLKklFRAA7xmAOk0M5jIR0t48UYAp2w== --osd-uuid 5b4ac019-50d1-4af2-8d30-7c88089dfed7 2026-02-17T12:17:43.485 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:43.483+0000 7f7bfaf93780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:43.486 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:43.485+0000 7f7bfaf93780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:43.487 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:43.486+0000 7f7bfaf93780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:43.487 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:43.486+0000 7f7bfaf93780 -1 bdev(0x558959addc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:17:43.487 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:43.486+0000 7f7bfaf93780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:17:45.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:17:45.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:17:45.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:17:45.613 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:17:45.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:17:45.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:17:45.884 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:17:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:17:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:17:45.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:17:45.900 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:45.898+0000 7f1d1ca22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:45.901 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:45.900+0000 7f1d1ca22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:45.902 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:45.901+0000 7f1d1ca22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:46.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:46.074 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:46.075 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:46.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:46.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:46.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:47.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:47.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:47.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:47.264 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:47.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:47.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:47.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:47.488 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:47.486+0000 7f1d1ca22780 -1 Falling back to public interface 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:48.453 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:48.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:49.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:49.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:49.644 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:49.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:49.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:49.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:49.835 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:50.438 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:50.436+0000 7f1d1ca22780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:17:50.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:50.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:50.837 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:17:50.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:50.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:50.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:51.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:52.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:52.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:52.029 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:17:52.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:17:52.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:52.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:52.221 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1336304244,v1:127.0.0.1:6803/1336304244] [v2:127.0.0.1:6804/1336304244,v1:127.0.0.1:6805/1336304244] exists,up 5b4ac019-50d1-4af2-8d30-7c88089dfed7 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:52.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:17:52.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:17:52.224 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 ae3180c7-eb02-44e4-b714-05cb6b504b58' 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:17:52.225 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:17:52.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBwXJRpv5AHDhAAJ2nlJ2ScVEYtkzWc18tc0A== 2026-02-17T12:17:52.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBwXJRpv5AHDhAAJ2nlJ2ScVEYtkzWc18tc0A=="}' 2026-02-17T12:17:52.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ae3180c7-eb02-44e4-b714-05cb6b504b58 -i td/test-erasure-eio/1/new.json 2026-02-17T12:17:52.426 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:52.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:17:52.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBwXJRpv5AHDhAAJ2nlJ2ScVEYtkzWc18tc0A== --osd-uuid ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:17:52.452 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:52.450+0000 7f50180d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:52.453 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:52.451+0000 7f50180d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:52.454 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:52.452+0000 7f50180d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:52.455 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:52.453+0000 7f50180d0780 -1 bdev(0x5628eecd7c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:17:52.455 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:52.453+0000 7f50180d0780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:17:54.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:17:54.816 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:17:54.817 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:17:54.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:17:54.818 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:17:55.068 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:17:55.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:17:55.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:55.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:17:55.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:17:55.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:17:55.086 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:55.085+0000 7f1686d16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:55.088 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:55.087+0000 7f1686d16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:55.088 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:55.087+0000 7f1686d16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:55.256 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:17:55.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:55.257 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:55.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:56.430 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:56.428+0000 7f1686d16780 -1 Falling back to public interface 2026-02-17T12:17:56.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:56.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:56.445 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:17:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:56.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:57.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:57.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:57.635 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:17:57.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:57.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:57.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:57.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:58.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:58.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:58.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:58.827 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:17:58.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:58.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:59.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:59.369 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:17:59.368+0000 7f1686d16780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:18:00.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:00.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:00.016 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:00.016 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:18:00.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:00.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:00.205 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1530688888,v1:127.0.0.1:6811/1530688888] [v2:127.0.0.1:6812/1530688888,v1:127.0.0.1:6813/1530688888] exists,up ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:18:00.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:00.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:00.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:00.211 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:18:00.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=61ea3726-81e2-44cd-9d47-68afc210b915 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 61ea3726-81e2-44cd-9d47-68afc210b915' 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 61ea3726-81e2-44cd-9d47-68afc210b915 2026-02-17T12:18:00.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:18:00.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB4XJRpxXwZDRAAFYs76w6bZpnFLw8X9Jd17g== 2026-02-17T12:18:00.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB4XJRpxXwZDRAAFYs76w6bZpnFLw8X9Jd17g=="}' 2026-02-17T12:18:00.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 61ea3726-81e2-44cd-9d47-68afc210b915 -i td/test-erasure-eio/2/new.json 2026-02-17T12:18:00.415 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:18:00.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:18:00.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB4XJRpxXwZDRAAFYs76w6bZpnFLw8X9Jd17g== --osd-uuid 61ea3726-81e2-44cd-9d47-68afc210b915 2026-02-17T12:18:00.442 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:00.441+0000 7fd26980f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:00.444 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:00.443+0000 7fd26980f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:00.445 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:00.444+0000 7fd26980f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:00.445 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:00.444+0000 7fd26980f780 -1 bdev(0x5650f248dc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:18:00.446 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:00.444+0000 7fd26980f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:18:02.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:18:02.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:18:02.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:18:02.570 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:18:02.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:18:02.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:18:02.840 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:02.838+0000 7f325faed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:02.842 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:02.840+0000 7f325faed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:02.842 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:02.840+0000 7f325faed780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:03.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:18:03.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:03.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:18:03.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:03.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:03.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:03.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:03.012 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:18:03.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:03.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:18:03.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:04.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:18:04.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:04.419 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:04.417+0000 7f325faed780 -1 Falling back to public interface 2026-02-17T12:18:05.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:05.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:05.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:05.389 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:18:05.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:05.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:18:05.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:06.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:06.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:06.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:06.579 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:18:06.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:06.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:18:06.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:07.358 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:07.356+0000 7f325faed780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:18:07.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:07.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:07.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:07.768 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:18:07.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:07.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:18:07.967 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/893698860,v1:127.0.0.1:6819/893698860] [v2:127.0.0.1:6820/893698860,v1:127.0.0.1:6821/893698860] exists,up 61ea3726-81e2-44cd-9d47-68afc210b915 2026-02-17T12:18:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:07.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:07.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:07.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:18:07.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:18:07.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:18:07.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:18:07.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:07.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 0c3c2788-e6f3-4a30-ad65-389d17d8c33d' 2026-02-17T12:18:07.975 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:07.975 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:18:07.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB/XJRpwXmHOhAAJ1ODN3a8pJpIyVi/Xplitw== 2026-02-17T12:18:07.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB/XJRpwXmHOhAAJ1ODN3a8pJpIyVi/Xplitw=="}' 2026-02-17T12:18:07.983 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0c3c2788-e6f3-4a30-ad65-389d17d8c33d -i td/test-erasure-eio/3/new.json 2026-02-17T12:18:08.174 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:18:08.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:18:08.183 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB/XJRpwXmHOhAAJ1ODN3a8pJpIyVi/Xplitw== --osd-uuid 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:08.200 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:08.199+0000 7f60767ad780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:08.202 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:08.199+0000 7f60767ad780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:08.203 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:08.200+0000 7f60767ad780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:08.203 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:08.201+0000 7f60767ad780 -1 bdev(0x562eb6d3bc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:18:08.203 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:08.201+0000 7f60767ad780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:18:10.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:18:10.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:18:10.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:18:10.315 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:18:10.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:18:10.570 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:18:10.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:18:10.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:10.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:18:10.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:18:10.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:18:10.587 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:10.586+0000 7fbf6d021780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:10.589 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:10.587+0000 7fbf6d021780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:10.590 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:10.588+0000 7fbf6d021780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:10.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:10.947 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:11.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:11.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:11.949 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:11.950 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:18:11.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:11.950 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:12.139 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:12.177 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:12.175+0000 7fbf6d021780 -1 Falling back to public interface 2026-02-17T12:18:13.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:13.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:13.141 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:13.142 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:18:13.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:13.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:13.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:14.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:14.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:14.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:14.329 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:18:14.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:14.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:14.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:15.136 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:15.134+0000 7fbf6d021780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:18:15.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:15.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:15.520 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:18:15.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:15.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:15.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:15.715 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:16.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:16.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:16.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:18:16.718 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:18:16.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:16.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:16.907 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3630497986,v1:127.0.0.1:6827/3630497986] [v2:127.0.0.1:6828/3630497986,v1:127.0.0.1:6829/3630497986] exists,up 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:16.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:18:16.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:18:16.909 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:18:16.955 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:18:16.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:18:16.961 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:17:47.489+0000 7f1d1ca22780 0 load: jerasure load: lrc 2026-02-17T12:18:16.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-17T12:18:16.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:18:16.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:18:16.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:18:16.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:18:16.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:18:16.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:18:16.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:18:16.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:18:17.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:18:17.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:18:17.505 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:18:17.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:18:18.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:18:18.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:18:18.598 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:18:18.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:18:18.788 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:18.788 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:18.788 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:18:18.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:18:18.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:18.789 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:18:18.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:18:18.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:18:18.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:18:18.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:18.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:18:18.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607559 2026-02-17T12:18:18.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607559 2026-02-17T12:18:18.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-17T12:18:18.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:18.911 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:18:18.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411334 2026-02-17T12:18:18.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411334 2026-02-17T12:18:18.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334' 2026-02-17T12:18:18.974 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:18.974 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:18:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964116995 2026-02-17T12:18:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964116995 2026-02-17T12:18:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334 3-115964116995' 2026-02-17T12:18:19.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:19.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:18:19.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:19.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:18:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:18:19.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:19.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:18:19.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:18:19.040 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:18:19.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:19.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:18:19.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:20.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:20.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:20.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-17T12:18:20.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:20.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607559 2026-02-17T12:18:20.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:20.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:20.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607559 2026-02-17T12:18:20.422 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:20.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607559 2026-02-17T12:18:20.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-17T12:18:20.423 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607559 2026-02-17T12:18:20.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:20.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-17T12:18:20.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:20.611 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411334 2026-02-17T12:18:20.611 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:20.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:18:20.613 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411334 2026-02-17T12:18:20.613 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:20.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411334 2026-02-17T12:18:20.614 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411334 2026-02-17T12:18:20.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-17T12:18:20.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:18:20.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-17T12:18:20.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:20.803 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-115964116995 2026-02-17T12:18:20.803 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:20.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:18:20.804 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-115964116995 2026-02-17T12:18:20.805 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:20.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964116995 2026-02-17T12:18:20.806 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 115964116995 2026-02-17T12:18:20.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-17T12:18:20.806 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:21.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-17T12:18:21.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:18:21.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:21.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:18:21.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:18:21.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:18:21.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:18:21.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:18:21.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:18:21.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:21.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=0 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=10 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=set 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-0-10 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-0-10 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:21.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:18:21.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:21.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:18:21.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-0-10 td/test-erasure-eio/ORIGINAL 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-0-10 td/test-erasure-eio 0 10 set 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-10 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=0 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=10 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-10 2026-02-17T12:18:21.713 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:21.714 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:18:21.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:18:21.907 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=3 2026-02-17T12:18:21.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:18:22.158 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:18:22.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:18:22.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 10 = 0 ']' 2026-02-17T12:18:22.165 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-17T12:18:22.167 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:10 bytes copied, 5.4561e-05 s, 183 kB/s 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:18:22.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:22.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:23.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-17T12:18:23.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:24.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-17T12:18:24.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:18:24.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:18:24.579 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:18:24.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:24.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:18:24.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:18:24.583 INFO:tasks.workunit.client.0.trial044.stderr:start osd.3 2026-02-17T12:18:24.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:24.583 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-17T12:18:24.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:18:24.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:18:24.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:18:24.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:18:24.594 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:24.593+0000 7fb425a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:24.594+0000 7fb425a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:24.595+0000 7fb425a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:18:24.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:24.773 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:18:24.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:24.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:24.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:25.934 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:25.932+0000 7fb425a2e780 -1 Falling back to public interface 2026-02-17T12:18:25.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:25.960 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:26.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:27.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:27.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:27.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:27.147 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:27.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:27.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:27.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:28.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:28.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:28.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:28.345 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:18:28.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:28.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:28.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:28.661 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:28.660+0000 7fb425a2e780 -1 osd.3 35 log_to_monitors true 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:29.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [27,35) [v2:127.0.0.1:6826/3847627705,v1:127.0.0.1:6827/3847627705] [v2:127.0.0.1:6828/3847627705,v1:127.0.0.1:6829/3847627705] exists,up 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:18:29.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:18:29.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:18:29.813 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:18:29.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:18:29.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:18:29.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:18:29.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:18:29.814 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:30.002 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:18:30.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803788 2026-02-17T12:18:30.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803788 2026-02-17T12:18:30.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788' 2026-02-17T12:18:30.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:30.063 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:18:30.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607563 2026-02-17T12:18:30.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607563 2026-02-17T12:18:30.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563' 2026-02-17T12:18:30.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:30.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:18:30.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411338 2026-02-17T12:18:30.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411338 2026-02-17T12:18:30.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-77309411338' 2026-02-17T12:18:30.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:30.188 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659139 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659139 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-77309411338 3-176093659139' 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803788 2026-02-17T12:18:30.251 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:30.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:18:30.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803788 2026-02-17T12:18:30.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:30.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803788 2026-02-17T12:18:30.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-17T12:18:30.254 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803788 2026-02-17T12:18:30.255 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:30.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-17T12:18:30.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:30.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607563 2026-02-17T12:18:30.451 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:30.452 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:30.452 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607563 2026-02-17T12:18:30.453 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:30.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607563 2026-02-17T12:18:30.454 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607563' 2026-02-17T12:18:30.454 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 51539607563 2026-02-17T12:18:30.454 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607563 -lt 51539607563 2026-02-17T12:18:30.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:30.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411338 2026-02-17T12:18:30.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:30.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:18:30.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411338 2026-02-17T12:18:30.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:30.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411338 2026-02-17T12:18:30.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-17T12:18:30.648 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411338 2026-02-17T12:18:30.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:18:30.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-17T12:18:30.836 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:30.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-176093659139 2026-02-17T12:18:30.836 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:30.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:18:30.838 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-176093659139 2026-02-17T12:18:30.838 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:30.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659139 2026-02-17T12:18:30.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 176093659139' 2026-02-17T12:18:30.839 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 176093659139 2026-02-17T12:18:30.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:31.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 176093659139 2026-02-17T12:18:31.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:32.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:32.028 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:32.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 176093659139 2026-02-17T12:18:32.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:33.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:18:33.218 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:33.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-17T12:18:33.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:18:33.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:33.410 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:33.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:18:33.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:18:33.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:18:33.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:18:33.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:18:33.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:33.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:18:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:18:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:18:34.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:18:34.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:18:34.375 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:18:34.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:34.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:18:34.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:18:34.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-10 2026-02-17T12:18:34.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:18:34.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:18:34.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-0-10 td/test-erasure-eio/COPY 2026-02-17T12:18:34.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:18:34.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:18:34.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 0 + 1 2026-02-17T12:18:34.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=1 2026-02-17T12:18:34.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-0-10 td/test-erasure-eio 1 10 set 2026-02-17T12:18:34.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-10 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=10 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:18:34.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-10 2026-02-17T12:18:34.410 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-10 2026-02-17T12:18:34.410 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:18:34.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:18:34.600 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:34.600 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:18:34.601 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:18:34.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:18:34.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:18:34.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:18:34.601 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:18:34.879 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:18:34.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:18:34.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 10 = 0 ']' 2026-02-17T12:18:34.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:10 bytes copied, 3.5365e-05 s, 283 kB/s 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:18:34.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:34.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:18:36.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:37.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:18:37.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:37.301 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:18:37.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:18:37.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:18:37.303 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:18:37.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:37.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:18:37.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:18:37.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:18:37.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:18:37.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:18:37.315 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:37.314+0000 7efd7f73d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:37.317 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:37.316+0000 7efd7f73d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:37.318 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:37.317+0000 7efd7f73d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:37.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:18:37.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:37.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:18:37.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:37.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:37.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:37.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:37.492 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:18:37.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:37.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:37.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:38.651 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:38.650+0000 7efd7f73d780 -1 Falling back to public interface 2026-02-17T12:18:38.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:38.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:38.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:38.679 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:38.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:38.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:38.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:39.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:40.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:41.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:41.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:41.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:41.061 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:18:41.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:41.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:41.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:41.352 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:41.349+0000 7efd7f73d780 -1 osd.1 47 log_to_monitors true 2026-02-17T12:18:42.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:42.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:42.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:42.252 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:18:42.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:42.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:42.438 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [12,47) [v2:127.0.0.1:6810/2697645561,v1:127.0.0.1:6811/2697645561] [v2:127.0.0.1:6812/2697645561,v1:127.0.0.1:6813/2697645561] exists,up ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:18:42.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:18:42.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:18:42.440 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:18:42.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:18:42.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:18:42.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:18:42.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:18:42.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:18:42.527 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:18:42.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:42.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:18:42.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803793 2026-02-17T12:18:42.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803793 2026-02-17T12:18:42.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793' 2026-02-17T12:18:42.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:42.779 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:18:42.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332099 2026-02-17T12:18:42.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332099 2026-02-17T12:18:42.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-219043332099' 2026-02-17T12:18:42.842 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:42.842 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:18:42.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411342 2026-02-17T12:18:42.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411342 2026-02-17T12:18:42.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-219043332099 2-77309411342' 2026-02-17T12:18:42.905 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:42.905 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:18:42.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659143 2026-02-17T12:18:42.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659143 2026-02-17T12:18:42.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-219043332099 2-77309411342 3-176093659143' 2026-02-17T12:18:42.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:42.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803793 2026-02-17T12:18:42.967 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:42.968 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:18:42.968 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803793 2026-02-17T12:18:42.968 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:42.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803793 2026-02-17T12:18:42.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-17T12:18:42.969 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803793 2026-02-17T12:18:42.969 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:43.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-17T12:18:43.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:44.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:44.159 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:44.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-17T12:18:44.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:44.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332099 2026-02-17T12:18:44.352 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:44.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:44.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332099 2026-02-17T12:18:44.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:44.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332099 2026-02-17T12:18:44.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332099' 2026-02-17T12:18:44.355 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 219043332099 2026-02-17T12:18:44.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:44.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332099 -lt 219043332099 2026-02-17T12:18:44.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:44.556 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411342 2026-02-17T12:18:44.556 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:44.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:18:44.558 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411342 2026-02-17T12:18:44.558 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:44.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411342 2026-02-17T12:18:44.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-17T12:18:44.559 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411342 2026-02-17T12:18:44.559 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:18:44.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411343 -lt 77309411342 2026-02-17T12:18:44.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:44.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-176093659143 2026-02-17T12:18:44.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:44.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:18:44.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-176093659143 2026-02-17T12:18:44.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:44.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659143 2026-02-17T12:18:44.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 176093659143' 2026-02-17T12:18:44.748 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 176093659143 2026-02-17T12:18:44.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:44.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659144 -lt 176093659143 2026-02-17T12:18:44.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:18:44.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:44.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:45.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:18:45.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:18:45.203 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:18:45.203 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:18:45.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:18:45.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:18:45.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:18:45.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:18:45.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:18:45.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:18:45.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:45.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:45.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:18:45.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:18:45.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:18:45.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:18:45.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:18:45.907 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:18:45.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-10 fail 2026-02-17T12:18:45.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:18:45.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:18:45.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-10 2026-02-17T12:18:45.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:18:45.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:18:45.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-0-10 td/test-erasure-eio/COPY 2026-02-17T12:18:45.936 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-0-10: (5) Input/output error 2026-02-17T12:18:45.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:18:45.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=set 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-0-0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-0-0 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:18:45.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:18:45.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:18:45.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-0-0 td/test-erasure-eio/ORIGINAL 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-0-0 td/test-erasure-eio 0 0 set 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-0 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=0 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=0 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:18:45.958 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-0 2026-02-17T12:18:45.959 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:45.959 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:18:46.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=3 2026-02-17T12:18:46.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:18:46.413 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:18:46.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:18:46.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 0 = 0 ']' 2026-02-17T12:18:46.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:18:46.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:46.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:47.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-17T12:18:47.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:18:48.822 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:48.823 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:18:48.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:18:48.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:18:48.826 INFO:tasks.workunit.client.0.trial044.stderr:start osd.3 2026-02-17T12:18:48.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:48.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-17T12:18:48.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:18:48.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:18:48.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:18:48.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:18:48.842 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:48.840+0000 7f5804b3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:48.843 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:48.842+0000 7f5804b3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:48.844 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:48.843+0000 7f5804b3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:49.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:49.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:49.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:50.171 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:50.170+0000 7f5804b3f780 -1 Falling back to public interface 2026-02-17T12:18:50.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:50.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:50.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:50.208 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:50.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:50.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:50.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:51.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:51.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:51.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:51.396 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:51.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:51.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:51.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:52.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:52.588 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:18:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:52.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:52.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:52.862 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:18:52.861+0000 7f5804b3f780 -1 osd.3 54 log_to_monitors true 2026-02-17T12:18:53.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:53.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:53.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:53.775 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:18:53.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:53.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:18:53.970 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [41,54) [v2:127.0.0.1:6826/197408063,v1:127.0.0.1:6827/197408063] [v2:127.0.0.1:6828/197408063,v1:127.0.0.1:6829/197408063] exists,up 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:18:53.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:18:53.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:18:53.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:18:53.972 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:18:53.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:18:53.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:18:53.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:18:54.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:18:54.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:18:54.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:18:54.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:18:54.059 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:18:54.059 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:18:54.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:54.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:18:54.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803797 2026-02-17T12:18:54.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803797 2026-02-17T12:18:54.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797' 2026-02-17T12:18:54.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:54.316 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:18:54.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332103 2026-02-17T12:18:54.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332103 2026-02-17T12:18:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-219043332103' 2026-02-17T12:18:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:54.381 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:18:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411347 2026-02-17T12:18:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411347 2026-02-17T12:18:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-219043332103 2-77309411347' 2026-02-17T12:18:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:54.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:18:54.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037763 2026-02-17T12:18:54.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037763 2026-02-17T12:18:54.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-219043332103 2-77309411347 3-257698037763' 2026-02-17T12:18:54.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:54.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803797 2026-02-17T12:18:54.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:54.508 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:18:54.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803797 2026-02-17T12:18:54.508 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:54.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803797 2026-02-17T12:18:54.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-17T12:18:54.510 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803797 2026-02-17T12:18:54.510 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:54.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-17T12:18:54.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:55.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:55.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:55.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-17T12:18:55.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:56.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:18:56.886 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:57.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-17T12:18:57.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:57.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332103 2026-02-17T12:18:57.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:57.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:57.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332103 2026-02-17T12:18:57.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:57.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332103 2026-02-17T12:18:57.081 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332103' 2026-02-17T12:18:57.081 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 219043332103 2026-02-17T12:18:57.081 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:57.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332103 -lt 219043332103 2026-02-17T12:18:57.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:57.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411347 2026-02-17T12:18:57.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:57.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:18:57.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411347 2026-02-17T12:18:57.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:57.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411347 2026-02-17T12:18:57.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411347' 2026-02-17T12:18:57.272 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411347 2026-02-17T12:18:57.272 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:18:57.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411347 -lt 77309411347 2026-02-17T12:18:57.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:57.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-257698037763 2026-02-17T12:18:57.462 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:57.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:18:57.464 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-257698037763 2026-02-17T12:18:57.464 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:57.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037763 2026-02-17T12:18:57.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 257698037763' 2026-02-17T12:18:57.465 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 257698037763 2026-02-17T12:18:57.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:18:57.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-17T12:18:57.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:18:57.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:57.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:18:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:18:57.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:18:58.098 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:18:58.098 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:18:58.099 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:58.099 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:58.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:18:58.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:18:58.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:18:58.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:18:58.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:18:58.639 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:18:58.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-0 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:18:58.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-0-0 td/test-erasure-eio/COPY 2026-02-17T12:18:58.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:18:58.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:18:58.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 0 + 1 2026-02-17T12:18:58.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=1 2026-02-17T12:18:58.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-0-0 td/test-erasure-eio 1 0 set 2026-02-17T12:18:58.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-0 2026-02-17T12:18:58.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:18:58.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:18:58.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:18:58.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=0 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:18:58.676 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:58.677 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:18:58.677 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-0 2026-02-17T12:18:58.677 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-0 2026-02-17T12:18:58.677 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:18:58.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:18:59.144 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:18:59.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:18:59.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 0 = 0 ']' 2026-02-17T12:18:59.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:18:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:59.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:00.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:00.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:00.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:19:00.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:19:00.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:19:00.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:19:00.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:00.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:01.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:19:01.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:19:01.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:01.558 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:01.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:01.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:01.561 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:19:01.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:19:01.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:01.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:01.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:01.574 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:01.572+0000 7f5a9c20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:01.575 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:01.574+0000 7f5a9c20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:01.576 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:01.575+0000 7f5a9c20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:01.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:19:01.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:01.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:19:01.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:01.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:01.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:02.914 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:02.913+0000 7f5a9c20f780 -1 Falling back to public interface 2026-02-17T12:19:02.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:02.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:02.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:02.935 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:02.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:02.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:03.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:04.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:04.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:05.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:05.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:05.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:05.323 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:19:05.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:05.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:05.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:05.623 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:05.621+0000 7f5a9c20f780 -1 osd.1 65 log_to_monitors true 2026-02-17T12:19:06.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:06.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:06.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:06.513 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:19:06.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:06.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:06.708 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [51,65) [v2:127.0.0.1:6810/3841251068,v1:127.0.0.1:6811/3841251068] [v2:127.0.0.1:6812/3841251068,v1:127.0.0.1:6813/3841251068] exists,up ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:19:06.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:06.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:06.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:06.709 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:06.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:06.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:06.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:06.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:06.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:06.797 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:06.985 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:07.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803802 2026-02-17T12:19:07.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803802 2026-02-17T12:19:07.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803802' 2026-02-17T12:19:07.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:07.046 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:07.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743427 2026-02-17T12:19:07.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743427 2026-02-17T12:19:07.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803802 1-296352743427' 2026-02-17T12:19:07.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:07.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:19:07.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411351 2026-02-17T12:19:07.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411351 2026-02-17T12:19:07.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803802 1-296352743427 2-77309411351' 2026-02-17T12:19:07.170 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:07.170 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:19:07.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037767 2026-02-17T12:19:07.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037767 2026-02-17T12:19:07.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803802 1-296352743427 2-77309411351 3-257698037767' 2026-02-17T12:19:07.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:07.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803802 2026-02-17T12:19:07.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:07.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:07.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803802 2026-02-17T12:19:07.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:07.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803802 2026-02-17T12:19:07.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-17T12:19:07.236 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803802 2026-02-17T12:19:07.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:07.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803799 -lt 25769803802 2026-02-17T12:19:07.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:08.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:08.432 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:08.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-17T12:19:08.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:08.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-296352743427 2026-02-17T12:19:08.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:08.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:08.622 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-296352743427 2026-02-17T12:19:08.622 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:08.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743427 2026-02-17T12:19:08.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 296352743427' 2026-02-17T12:19:08.623 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 296352743427 2026-02-17T12:19:08.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:08.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743427 -lt 296352743427 2026-02-17T12:19:08.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:08.814 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411351 2026-02-17T12:19:08.815 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:08.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:19:08.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411351 2026-02-17T12:19:08.816 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:08.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411351 2026-02-17T12:19:08.817 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411351' 2026-02-17T12:19:08.817 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411351 2026-02-17T12:19:08.818 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:19:09.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411351 -lt 77309411351 2026-02-17T12:19:09.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:09.008 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-257698037767 2026-02-17T12:19:09.008 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:09.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:19:09.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-257698037767 2026-02-17T12:19:09.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:09.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037767 2026-02-17T12:19:09.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 257698037767' 2026-02-17T12:19:09.011 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 257698037767 2026-02-17T12:19:09.012 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:19:09.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037768 -lt 257698037767 2026-02-17T12:19:09.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:09.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:09.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:09.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:19:09.455 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:09.455 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:09.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:09.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:09.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:09.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:09.456 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:09.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:19:09.640 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:09.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:09.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:09.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:19:09.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:09.894 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:09.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:19:09.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:19:10.159 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:19:10.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-0 fail 2026-02-17T12:19:10.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:19:10.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:19:10.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-0 2026-02-17T12:19:10.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:19:10.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:19:10.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-0-0 td/test-erasure-eio/COPY 2026-02-17T12:19:10.186 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-0-0: (5) Input/output error 2026-02-17T12:19:10.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:19:10.187 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:311: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=0 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:19:10.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=256 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=add 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-0-256 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-0-256 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:19:10.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:19:10.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:19:10.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:19:10.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:19:10.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:19:10.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-0-256 td/test-erasure-eio/ORIGINAL 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-0-256 td/test-erasure-eio 0 256 add 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-256 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=0 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:19:10.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=256 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=add 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-256 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:10.210 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:19:10.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:19:10.397 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:10.397 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:19:10.397 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:19:10.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:19:10.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:19:10.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=3 2026-02-17T12:19:10.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:19:10.665 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:19:10.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' add = add ']' 2026-02-17T12:19:10.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:10.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:19:10.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:19:10.674 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:10.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:11.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-17T12:19:11.982 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:12.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:12.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:12.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:19:12.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:19:12.802 INFO:tasks.workunit.client.0.trial044.stderr:start osd.3 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:12.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:12.815 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:12.814+0000 7fdeb65a9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:12.817 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:12.816+0000 7fdeb65a9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:12.818 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:12.817+0000 7fdeb65a9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:19:12.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:12.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:13.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:14.154 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:14.153+0000 7fdeb65a9780 -1 Falling back to public interface 2026-02-17T12:19:14.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:14.179 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:14.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:14.180 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:14.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:14.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:14.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:15.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:15.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:15.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:15.367 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:15.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:15.367 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:15.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:16.556 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:16.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:16.895 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:16.894+0000 7fdeb65a9780 -1 osd.3 72 log_to_monitors true 2026-02-17T12:19:17.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:17.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:17.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:17.745 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:19:17.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:17.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [60,72) [v2:127.0.0.1:6826/2732303579,v1:127.0.0.1:6827/2732303579] [v2:127.0.0.1:6828/2732303579,v1:127.0.0.1:6829/2732303579] exists,up 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:17.932 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:17.933 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:17.933 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:17.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:17.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:17.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:18.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:18.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803806 2026-02-17T12:19:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803806 2026-02-17T12:19:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806' 2026-02-17T12:19:18.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:18.272 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:18.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743431 2026-02-17T12:19:18.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743431 2026-02-17T12:19:18.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-296352743431' 2026-02-17T12:19:18.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:18.335 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:19:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411355 2026-02-17T12:19:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411355 2026-02-17T12:19:18.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-296352743431 2-77309411355' 2026-02-17T12:19:18.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:18.396 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:19:18.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449091 2026-02-17T12:19:18.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449091 2026-02-17T12:19:18.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-296352743431 2-77309411355 3-335007449091' 2026-02-17T12:19:18.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:18.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803806 2026-02-17T12:19:18.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:18.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:18.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803806 2026-02-17T12:19:18.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:18.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803806 2026-02-17T12:19:18.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803806' 2026-02-17T12:19:18.460 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803806 2026-02-17T12:19:18.460 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803804 -lt 25769803806 2026-02-17T12:19:18.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:19.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:19.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:19.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803804 -lt 25769803806 2026-02-17T12:19:19.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:20.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:19:20.841 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:21.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803806 -lt 25769803806 2026-02-17T12:19:21.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:21.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-296352743431 2026-02-17T12:19:21.033 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:21.034 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:21.035 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-296352743431 2026-02-17T12:19:21.035 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:21.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743431 2026-02-17T12:19:21.036 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 296352743431' 2026-02-17T12:19:21.036 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 296352743431 2026-02-17T12:19:21.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:21.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743431 -lt 296352743431 2026-02-17T12:19:21.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:21.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411355 2026-02-17T12:19:21.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:21.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:19:21.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411355 2026-02-17T12:19:21.224 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:21.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411355 2026-02-17T12:19:21.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411355' 2026-02-17T12:19:21.225 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411355 2026-02-17T12:19:21.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:19:21.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411356 -lt 77309411355 2026-02-17T12:19:21.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:21.413 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-335007449091 2026-02-17T12:19:21.413 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:21.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:19:21.414 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-335007449091 2026-02-17T12:19:21.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:21.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449091 2026-02-17T12:19:21.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 335007449091' 2026-02-17T12:19:21.416 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 335007449091 2026-02-17T12:19:21.416 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:19:21.606 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-17T12:19:21.606 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:21.606 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:21.607 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:21.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:19:21.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:21.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:21.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:21.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:21.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:21.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:21.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:22.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:19:22.051 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:22.052 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:22.052 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:22.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:19:22.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:22.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:22.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:256 bytes copied, 2.162e-05 s, 11.8 MB/s 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:19:22.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:22.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:22.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:23.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:23.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:23.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:19:23.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:19:23.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:19:23.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:19:23.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-17T12:19:23.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:24.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:24.736 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:start osd.3 2026-02-17T12:19:24.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:24.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-17T12:19:24.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:19:24.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:24.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:24.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:24.758 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:24.755+0000 7fc774497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:24.759 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:24.757+0000 7fc774497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:24.760 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:24.757+0000 7fc774497780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:19:24.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:24.932 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:25.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:26.110 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:26.108+0000 7fc774497780 -1 Falling back to public interface 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:26.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:26.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:27.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:27.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:27.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:27.315 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:27.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:27.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:27.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:28.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:28.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:28.813 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:28.811+0000 7fc774497780 -1 osd.3 80 log_to_monitors true 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:29.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:19:29.898 INFO:tasks.workunit.client.0.trial044.stderr:osd.3 up in weight 1 up_from 84 up_thru 84 down_at 81 last_clean_interval [78,80) [v2:127.0.0.1:6826/2455925432,v1:127.0.0.1:6827/2455925432] [v2:127.0.0.1:6828/2455925432,v1:127.0.0.1:6829/2455925432] exists,up 0c3c2788-e6f3-4a30-ad65-389d17d8c33d 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:29.899 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:29.900 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:29.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:29.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:29.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:29.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:29.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:29.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:29.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:29.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:29.987 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:30.177 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:30.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803810 2026-02-17T12:19:30.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803810 2026-02-17T12:19:30.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810' 2026-02-17T12:19:30.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:30.240 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:30.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743435 2026-02-17T12:19:30.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743435 2026-02-17T12:19:30.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-296352743435' 2026-02-17T12:19:30.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:30.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:19:30.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411360 2026-02-17T12:19:30.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411360 2026-02-17T12:19:30.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-296352743435 2-77309411360' 2026-02-17T12:19:30.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:30.367 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:19:30.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=360777252867 2026-02-17T12:19:30.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 360777252867 2026-02-17T12:19:30.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-296352743435 2-77309411360 3-360777252867' 2026-02-17T12:19:30.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:30.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803810 2026-02-17T12:19:30.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:30.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:30.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803810 2026-02-17T12:19:30.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:30.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803810 2026-02-17T12:19:30.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803810' 2026-02-17T12:19:30.432 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803810 2026-02-17T12:19:30.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:30.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803808 -lt 25769803810 2026-02-17T12:19:30.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:31.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:31.621 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:31.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803808 -lt 25769803810 2026-02-17T12:19:31.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:32.811 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:19:32.811 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:33.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803811 -lt 25769803810 2026-02-17T12:19:33.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:33.008 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-296352743435 2026-02-17T12:19:33.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:33.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:33.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-296352743435 2026-02-17T12:19:33.010 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:33.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743435 2026-02-17T12:19:33.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 296352743435' 2026-02-17T12:19:33.011 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 296352743435 2026-02-17T12:19:33.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:33.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743436 -lt 296352743435 2026-02-17T12:19:33.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:33.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411360 2026-02-17T12:19:33.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:33.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:19:33.203 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411360 2026-02-17T12:19:33.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:33.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411360 2026-02-17T12:19:33.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411360' 2026-02-17T12:19:33.205 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411360 2026-02-17T12:19:33.205 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:19:33.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411360 -lt 77309411360 2026-02-17T12:19:33.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:33.398 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-360777252867 2026-02-17T12:19:33.398 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:33.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:19:33.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-360777252867 2026-02-17T12:19:33.400 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:33.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=360777252867 2026-02-17T12:19:33.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 360777252867' 2026-02-17T12:19:33.401 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 360777252867 2026-02-17T12:19:33.401 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:19:33.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 360777252867 -lt 360777252867 2026-02-17T12:19:33.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:33.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:33.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:33.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:19:33.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:33.850 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:33.850 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:33.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:33.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:33.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:33.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:34.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:19:34.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:34.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:34.037 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:34.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:19:34.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:34.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:34.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:19:34.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:19:34.552 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:19:34.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-256 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:19:34.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-0-256 td/test-erasure-eio/COPY 2026-02-17T12:19:34.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:19:34.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:19:34.584 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 0 + 1 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=1 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-0-256 td/test-erasure-eio 1 256 add 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-0-256 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:19:34.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=256 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=add 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-0-256 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-0-256 2026-02-17T12:19:34.588 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:19:34.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:19:35.055 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:19:35.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' add = add ']' 2026-02-17T12:19:35.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:19:35.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:35.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:36.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:36.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:19:36.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:19:36.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:36.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:36.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:19:36.938 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:36.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:36.952 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:36.951+0000 7f04a08c8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:36.954 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:36.952+0000 7f04a08c8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:36.955 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:36.953+0000 7f04a08c8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:37.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:37.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:38.286 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:38.284+0000 7f04a08c8780 -1 Falling back to public interface 2026-02-17T12:19:38.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:38.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:38.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:38.316 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:38.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:38.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:38.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:39.517 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:39.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:39.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:39.518 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:39.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:39.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:39.708 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:40.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:40.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:40.979 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:40.977+0000 7f04a08c8780 -1 osd.1 88 log_to_monitors true 2026-02-17T12:19:41.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:41.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:41.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:41.901 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:19:41.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:41.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:42.091 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 94 up_thru 94 down_at 91 last_clean_interval [69,88) [v2:127.0.0.1:6810/2271039048,v1:127.0.0.1:6811/2271039048] [v2:127.0.0.1:6812/2271039048,v1:127.0.0.1:6813/2271039048] exists,up ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:42.092 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:42.093 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:42.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:42.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:42.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:42.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:42.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:42.181 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:42.375 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:42.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803815 2026-02-17T12:19:42.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803815 2026-02-17T12:19:42.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803815' 2026-02-17T12:19:42.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:42.440 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:42.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=403726925827 2026-02-17T12:19:42.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 403726925827 2026-02-17T12:19:42.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803815 1-403726925827' 2026-02-17T12:19:42.502 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:42.502 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:19:42.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411364 2026-02-17T12:19:42.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411364 2026-02-17T12:19:42.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803815 1-403726925827 2-77309411364' 2026-02-17T12:19:42.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:42.566 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:19:42.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=360777252871 2026-02-17T12:19:42.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 360777252871 2026-02-17T12:19:42.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803815 1-403726925827 2-77309411364 3-360777252871' 2026-02-17T12:19:42.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:42.628 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803815 2026-02-17T12:19:42.628 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:42.628 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:42.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803815 2026-02-17T12:19:42.629 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:42.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803815 2026-02-17T12:19:42.630 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803815' 2026-02-17T12:19:42.630 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803815 2026-02-17T12:19:42.631 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:42.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803813 -lt 25769803815 2026-02-17T12:19:42.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:43.825 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:43.826 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:44.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803813 -lt 25769803815 2026-02-17T12:19:44.017 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:45.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:19:45.018 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:45.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803815 -lt 25769803815 2026-02-17T12:19:45.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:45.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-403726925827 2026-02-17T12:19:45.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:45.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:45.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-403726925827 2026-02-17T12:19:45.213 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:45.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=403726925827 2026-02-17T12:19:45.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 403726925827' 2026-02-17T12:19:45.215 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 403726925827 2026-02-17T12:19:45.215 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:45.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 403726925827 -lt 403726925827 2026-02-17T12:19:45.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:45.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411364 2026-02-17T12:19:45.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:45.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:19:45.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411364 2026-02-17T12:19:45.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:45.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411364 2026-02-17T12:19:45.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411364' 2026-02-17T12:19:45.407 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411364 2026-02-17T12:19:45.408 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:19:45.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411365 -lt 77309411364 2026-02-17T12:19:45.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:45.593 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-360777252871 2026-02-17T12:19:45.593 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:45.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:19:45.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-360777252871 2026-02-17T12:19:45.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:45.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=360777252871 2026-02-17T12:19:45.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 360777252871' 2026-02-17T12:19:45.596 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 360777252871 2026-02-17T12:19:45.597 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:19:45.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 360777252872 -lt 360777252871 2026-02-17T12:19:45.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:45.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:45.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:46.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:46.043 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:46.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:19:46.232 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:46.233 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:46.233 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:46.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:19:46.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:46.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:46.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:256 bytes copied, 8.5929e-05 s, 3.0 MB/s 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:46.488 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:46.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:47.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:19:48.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:48.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:48.888 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:19:48.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:19:48.889 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:19:48.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:48.889 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:19:48.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:19:48.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:48.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:48.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:48.905 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:48.904+0000 7fa18252a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:48.906 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:48.905+0000 7fa18252a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:48.907 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:48.906+0000 7fa18252a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:49.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:19:49.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:49.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:19:49.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:49.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:49.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:50.242+0000 7fa18252a780 -1 Falling back to public interface 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:50.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:51.465 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:51.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:52.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:52.846 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:52.926 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:19:52.924+0000 7fa18252a780 -1 osd.1 95 log_to_monitors true 2026-02-17T12:19:53.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:53.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:53.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:53.848 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:19:53.848 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:53.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:54.037 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [94,95) [v2:127.0.0.1:6810/3655187046,v1:127.0.0.1:6811/3655187046] [v2:127.0.0.1:6812/3655187046,v1:127.0.0.1:6813/3655187046] exists,up ae3180c7-eb02-44e4-b714-05cb6b504b58 2026-02-17T12:19:54.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:54.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:54.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:54.037 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:54.038 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:54.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:54.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:54.039 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:54.124 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:54.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803819 2026-02-17T12:19:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803819 2026-02-17T12:19:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803819' 2026-02-17T12:19:54.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:54.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762307 2026-02-17T12:19:54.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762307 2026-02-17T12:19:54.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803819 1-425201762307' 2026-02-17T12:19:54.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:54.444 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:19:54.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411369 2026-02-17T12:19:54.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411369 2026-02-17T12:19:54.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803819 1-425201762307 2-77309411369' 2026-02-17T12:19:54.507 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:54.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:19:54.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=360777252876 2026-02-17T12:19:54.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 360777252876 2026-02-17T12:19:54.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803819 1-425201762307 2-77309411369 3-360777252876' 2026-02-17T12:19:54.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:54.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803819 2026-02-17T12:19:54.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:54.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:54.571 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803819 2026-02-17T12:19:54.571 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:54.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803819 2026-02-17T12:19:54.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803819' 2026-02-17T12:19:54.572 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803819 2026-02-17T12:19:54.572 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:54.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803817 -lt 25769803819 2026-02-17T12:19:54.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:55.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:55.766 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:55.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803817 -lt 25769803819 2026-02-17T12:19:55.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:56.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:19:56.959 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:57.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803819 -lt 25769803819 2026-02-17T12:19:57.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:57.148 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-425201762307 2026-02-17T12:19:57.149 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:57.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:57.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-425201762307 2026-02-17T12:19:57.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:57.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762307 2026-02-17T12:19:57.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 425201762307' 2026-02-17T12:19:57.151 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 425201762307 2026-02-17T12:19:57.151 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:57.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762307 -lt 425201762307 2026-02-17T12:19:57.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:57.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411369 2026-02-17T12:19:57.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:57.345 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:19:57.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411369 2026-02-17T12:19:57.346 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:57.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411369 2026-02-17T12:19:57.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411369' 2026-02-17T12:19:57.347 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411369 2026-02-17T12:19:57.347 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:19:57.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411369 -lt 77309411369 2026-02-17T12:19:57.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:57.537 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-360777252876 2026-02-17T12:19:57.537 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:57.538 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:19:57.538 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-360777252876 2026-02-17T12:19:57.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:57.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=360777252876 2026-02-17T12:19:57.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 360777252876' 2026-02-17T12:19:57.540 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 360777252876 2026-02-17T12:19:57.540 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:19:57.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 360777252876 -lt 360777252876 2026-02-17T12:19:57.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:57.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:57.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:57.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:19:57.978 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:57.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:58.166 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:19:58.166 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:58.167 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:58.167 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:58.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:19:58.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:58.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:58.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:19:58.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:19:58.721 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:19:58.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-0-256 fail 2026-02-17T12:19:58.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:19:58.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:19:58.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-0-256 2026-02-17T12:19:58.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:19:58.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:19:58.733 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-0-256 td/test-erasure-eio/COPY 2026-02-17T12:19:58.752 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-0-256: (5) Input/output error 2026-02-17T12:19:58.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:19:58.753 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:19:58.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:312: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:19:58.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:19:58.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:19:58.973 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:19:58.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:19:59.179 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:59.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:59.189 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:59.297 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:59.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:19:59.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:19:59.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:19:59.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:19:59.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:19:59.300 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:19:59.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:59.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:19:59.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:19:59.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:59.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:19:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:19:59.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:19:59.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:19:59.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:19:59.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:19:59.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:59.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:59.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:59.324 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:19:59.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:19:59.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:19:59.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:19:59.326 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:19:59.327 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:19:59.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:59.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:19:59.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:19:59.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:59.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:19:59.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:19:59.330 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:19:59.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:19:59.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.331 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:19:59.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:19:59.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:19:59.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:19:59.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:19:59.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:19:59.335 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:19:59.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:19:59.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:59.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:59.356 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:59.356 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.356 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:59.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:19:59.381 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:19:59.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:19:59.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:19:59.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:19:59.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:19:59.382 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:19:59.383 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:19:59.383 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:19:59.383 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:19:59.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:19:59.428 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:19:59.428 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:19:59.428 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:19:59.428 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:19:59.428 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:19:59.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:19:59.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:19:59.429 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:19:59.430 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:19:59.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:19:59.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:19:59.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:19:59.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:19:59.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:19:59.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:19:59.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:19:59.572 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:19:59.573 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:19:59.574 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:19:59.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:19:59.590 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:19:59.591 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:19:59.684 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:19:59.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:20:00.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:20:00.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:20:00.693 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:20:00.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:20:00.735 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:20:00.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:20:00.741 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:19:59.373+0000 7fb1e67d4d80 0 load: jerasure load: lrc 2026-02-17T12:20:00.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-17T12:20:00.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:316: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-17T12:20:00.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:317: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-17T12:20:00.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:20:00.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:20:00.742 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:20:00.743 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:20:00.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:20:00.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:20:00.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:00.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:20:00.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:00.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1a183a35-4432-49d9-9830-a7b453c46f13 2026-02-17T12:20:00.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1a183a35-4432-49d9-9830-a7b453c46f13' 2026-02-17T12:20:00.750 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 1a183a35-4432-49d9-9830-a7b453c46f13 2026-02-17T12:20:00.751 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:00.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDwXJRpB2g0LRAA1mQdes9G1NWreBn+K0Uq/A== 2026-02-17T12:20:00.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDwXJRpB2g0LRAA1mQdes9G1NWreBn+K0Uq/A=="}' 2026-02-17T12:20:00.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1a183a35-4432-49d9-9830-a7b453c46f13 -i td/test-erasure-eio/0/new.json 2026-02-17T12:20:00.862 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:20:00.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:20:00.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwXJRpB2g0LRAA1mQdes9G1NWreBn+K0Uq/A== --osd-uuid 1a183a35-4432-49d9-9830-a7b453c46f13 2026-02-17T12:20:00.890 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:00.887+0000 7fd1d322e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:00.891 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:00.889+0000 7fd1d322e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:00.892 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:00.890+0000 7fd1d322e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:00.893 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:00.890+0000 7fd1d322e780 -1 bdev(0x55af83103c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:00.893 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:00.890+0000 7fd1d322e780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:20:03.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:20:03.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:03.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:20:03.260 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:20:03.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:03.520 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:20:03.521 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:20:03.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:03.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:03.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:03.522 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:03.538 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:03.536+0000 7f8935353780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:03.539 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:03.537+0000 7f8935353780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:03.540 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:03.538+0000 7f8935353780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:03.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:20:03.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:03.713 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:20:03.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:03.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:03.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:04.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:04.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:04.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:04.901 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:20:04.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:04.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:05.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:05.135 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:05.132+0000 7f8935353780 -1 Falling back to public interface 2026-02-17T12:20:06.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:06.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:06.095 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:20:06.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:06.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:06.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:06.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:07.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:07.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:07.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:07.285 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:20:07.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:07.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:07.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:08.077 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:08.074+0000 7f8935353780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:20:08.473 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:08.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:08.474 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:20:08.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:08.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:08.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:08.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:09.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:09.860 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1817545211,v1:127.0.0.1:6803/1817545211] [v2:127.0.0.1:6804/1817545211,v1:127.0.0.1:6805/1817545211] exists,up 1a183a35-4432-49d9-9830-a7b453c46f13 2026-02-17T12:20:09.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:09.865 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:09.866 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:09.867 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 48af8c38-1c26-42fa-a196-d2d8888edbc3' 2026-02-17T12:20:09.868 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:20:09.869 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:09.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD5XJRpA/4qNBAApBlUIvfbYSOGbTnfr9eZwg== 2026-02-17T12:20:09.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD5XJRpA/4qNBAApBlUIvfbYSOGbTnfr9eZwg=="}' 2026-02-17T12:20:09.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 48af8c38-1c26-42fa-a196-d2d8888edbc3 -i td/test-erasure-eio/1/new.json 2026-02-17T12:20:10.069 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:20:10.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:20:10.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5XJRpA/4qNBAApBlUIvfbYSOGbTnfr9eZwg== --osd-uuid 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:20:10.094 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:10.092+0000 7fbcc4b09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:10.095 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:10.094+0000 7fbcc4b09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:10.096 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:10.095+0000 7fbcc4b09780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:10.096 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:10.095+0000 7fbcc4b09780 -1 bdev(0x55dbb83f3c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:10.097 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:10.095+0000 7fbcc4b09780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:20:12.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:20:12.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:12.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:20:12.216 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:20:12.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:12.475 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:20:12.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:20:12.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:12.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:12.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:12.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:12.492 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:12.490+0000 7fb10f427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:12.493 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:12.492+0000 7fb10f427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:12.494 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:12.492+0000 7fb10f427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:12.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:20:12.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:12.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:20:12.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:12.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:12.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:12.666 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:20:12.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:12.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:12.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:12.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:13.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:13.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:13.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:13.858 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:20:13.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:13.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:14.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:14.074 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:14.072+0000 7fb10f427780 -1 Falling back to public interface 2026-02-17T12:20:15.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:15.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:15.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:15.048 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:20:15.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:15.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:15.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:16.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:16.238 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:20:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:16.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:16.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:17.020 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.019+0000 7fb10f427780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:20:17.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:17.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:17.426 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:20:17.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:17.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:17.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:17.616 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1083310818,v1:127.0.0.1:6811/1083310818] [v2:127.0.0.1:6812/1083310818,v1:127.0.0.1:6813/1083310818] exists,up 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:20:17.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:17.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:17.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:17.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:20:17.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:20:17.622 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:17.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:17.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:20:17.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 b2ab1ec0-094b-4965-b566-a37328a0bf92' 2026-02-17T12:20:17.625 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:20:17.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:17.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQABXZRpieeWJRAA6IGe26vVh0XMlUO4McX+eA== 2026-02-17T12:20:17.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQABXZRpieeWJRAA6IGe26vVh0XMlUO4McX+eA=="}' 2026-02-17T12:20:17.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b2ab1ec0-094b-4965-b566-a37328a0bf92 -i td/test-erasure-eio/2/new.json 2026-02-17T12:20:17.823 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:20:17.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:20:17.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQABXZRpieeWJRAA6IGe26vVh0XMlUO4McX+eA== --osd-uuid b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:20:17.848 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.846+0000 7f141feba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.849 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.848+0000 7f141feba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.850 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.849+0000 7f141feba780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.851 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.849+0000 7f141feba780 -1 bdev(0x558d8d6c5c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:17.851 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:17.849+0000 7f141feba780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:20:19.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:20:19.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:19.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:20:19.966 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:20:19.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:20.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:20:20.219 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:20:20.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:20.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:20.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:20.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:20.236 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:20.234+0000 7f998d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:20.237 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:20.236+0000 7f998d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:20.238 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:20.237+0000 7f998d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:20.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:20:20.411 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:20:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:20.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:20.412 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:20:20.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:20.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:20.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:20.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:21.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:21.602 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:20:21.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:21.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:21.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:21.822 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:21.821+0000 7f998d619780 -1 Falling back to public interface 2026-02-17T12:20:22.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:22.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:22.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:22.789 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:20:22.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:22.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:22.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:23.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:23.980 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:23.981 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:20:23.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:23.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:23.981 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:24.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:24.763 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:24.761+0000 7f998d619780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:20:25.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:25.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:25.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:25.173 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:20:25.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:25.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:25.369 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1947534748,v1:127.0.0.1:6819/1947534748] [v2:127.0.0.1:6820/1947534748,v1:127.0.0.1:6821/1947534748] exists,up b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:20:25.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:25.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:25.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:20:25.378 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:25.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:20:25.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:25.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:25.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:25.379 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:25.380 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:25.381 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:25.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:25.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:25.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:25.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:25.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:25.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:20:25.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:20:25.384 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:25.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ec653f19-bd6d-4ea5-b2ad-a5735c2ef949 2026-02-17T12:20:25.384 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 ec653f19-bd6d-4ea5-b2ad-a5735c2ef949 2026-02-17T12:20:25.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 ec653f19-bd6d-4ea5-b2ad-a5735c2ef949' 2026-02-17T12:20:25.386 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:25.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAJXZRpdmXTFhAA+pbBzqBfixl85LjaQXtrmw== 2026-02-17T12:20:25.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAJXZRpdmXTFhAA+pbBzqBfixl85LjaQXtrmw=="}' 2026-02-17T12:20:25.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ec653f19-bd6d-4ea5-b2ad-a5735c2ef949 -i td/test-erasure-eio/3/new.json 2026-02-17T12:20:25.576 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:20:25.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:20:25.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAJXZRpdmXTFhAA+pbBzqBfixl85LjaQXtrmw== --osd-uuid ec653f19-bd6d-4ea5-b2ad-a5735c2ef949 2026-02-17T12:20:25.603 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:25.601+0000 7ff21f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:25.604 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:25.603+0000 7ff21f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:25.605 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:25.604+0000 7ff21f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:25.605 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:25.604+0000 7ff21f40d780 -1 bdev(0x55fe6f72bc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:25.606 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:25.604+0000 7ff21f40d780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:20:28.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:20:28.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:28.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:20:28.223 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:20:28.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:28.476 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:20:28.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:20:28.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:28.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:28.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:28.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:28.492 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:28.491+0000 7f046b5b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:28.494 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:28.492+0000 7f046b5b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:28.495 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:28.493+0000 7f046b5b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:28.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:28.666 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:20:28.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:28.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:28.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:20:28.856 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:29.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:29.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:29.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:29.858 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:20:29.858 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:29.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:20:30.047 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:30.081 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:30.080+0000 7f046b5b4780 -1 Falling back to public interface 2026-02-17T12:20:31.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:31.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:31.049 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:20:31.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:31.049 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:31.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:20:31.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:32.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:32.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:32.239 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:20:32.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:32.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:32.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:20:32.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:32.786 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:32.785+0000 7f046b5b4780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:20:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:33.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:33.430 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:20:33.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:33.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:20:33.615 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/25587187,v1:127.0.0.1:6827/25587187] [v2:127.0.0.1:6828/25587187,v1:127.0.0.1:6829/25587187] exists,up ec653f19-bd6d-4ea5-b2ad-a5735c2ef949 2026-02-17T12:20:33.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:33.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:33.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:33.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:20:33.617 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:20:33.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:20:33.661 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:20:33.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:20:33.667 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:20:05.136+0000 7f8935353780 0 load: jerasure load: lrc 2026-02-17T12:20:33.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:319: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-17T12:20:33.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:320: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:20:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:20:33.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:20:33.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:20:34.211 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:20:34.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:20:35.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:20:35.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:20:35.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:20:35.222 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:20:35.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:20:35.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:20:35.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:20:35.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:20:35.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:20:35.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:35.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:35.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:35.495 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:35.496 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:20:35.496 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:20:35.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:35.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:35.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:35.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:20:35.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:20:35.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:20:35.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:35.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:35.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607558 2026-02-17T12:20:35.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607558 2026-02-17T12:20:35.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558' 2026-02-17T12:20:35.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:35.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:35.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411332 2026-02-17T12:20:35.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411332 2026-02-17T12:20:35.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-77309411332' 2026-02-17T12:20:35.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:35.682 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-77309411332 3-111669149699' 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:20:35.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:35.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:35.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:20:35.747 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:35.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:20:35.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:20:35.748 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:20:35.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:35.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-17T12:20:35.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:35.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607558 2026-02-17T12:20:35.938 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:35.938 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:35.939 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607558 2026-02-17T12:20:35.939 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:35.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607558 2026-02-17T12:20:35.940 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607558 2026-02-17T12:20:35.941 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-17T12:20:35.941 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:36.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607556 -lt 51539607558 2026-02-17T12:20:36.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:37.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:37.132 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:37.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607556 -lt 51539607558 2026-02-17T12:20:37.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:38.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:20:38.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:38.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-17T12:20:38.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:38.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411332 2026-02-17T12:20:38.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:38.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:38.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411332 2026-02-17T12:20:38.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:38.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411332 2026-02-17T12:20:38.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-17T12:20:38.515 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411332 2026-02-17T12:20:38.515 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:38.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-17T12:20:38.701 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:38.702 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149699 2026-02-17T12:20:38.702 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:38.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:20:38.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149699 2026-02-17T12:20:38.703 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:38.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T12:20:38.704 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149699 2026-02-17T12:20:38.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-17T12:20:38.705 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:20:38.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T12:20:38.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:20:38.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:38.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:20:39.156 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:20:39.157 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:20:39.157 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:20:39.344 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:20:39.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:20:39.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:39.345 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=1 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=10 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=set 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-1-10 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-1-10 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:20:39.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:20:39.599 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-1-10 td/test-erasure-eio/ORIGINAL 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-1-10 td/test-erasure-eio 1 10 set 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-10 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:20:39.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=10 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-10 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:39.620 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:20:39.809 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:20:39.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:20:40.064 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:20:40.072 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:20:40.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 10 = 0 ']' 2026-02-17T12:20:40.073 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:10 bytes copied, 4.885e-05 s, 205 kB/s 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:20:40.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:20:40.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:20:40.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:20:41.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:20:41.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:41.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:20:41.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:20:41.384 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:20:41.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:20:41.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:20:41.385 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:42.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:20:42.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:42.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:20:42.481 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:20:42.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:42.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:42.484 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:20:42.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:20:42.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:20:42.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:20:42.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:20:42.497 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:42.494+0000 7fd35d613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:42.498 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:42.496+0000 7fd35d613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:42.499 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:42.496+0000 7fd35d613780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:42.671 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:20:42.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:42.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:42.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:43.579 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:43.576+0000 7fd35d613780 -1 Falling back to public interface 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:43.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:44.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:45.045 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:45.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:46.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:46.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:46.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:46.239 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:20:46.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:46.240 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:46.279 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:46.277+0000 7fd35d613780 -1 osd.1 32 log_to_monitors true 2026-02-17T12:20:46.427 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:47.430 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 38 up_thru 38 down_at 35 last_clean_interval [12,32) [v2:127.0.0.1:6810/3687895811,v1:127.0.0.1:6811/3687895811] [v2:127.0.0.1:6812/3687895811,v1:127.0.0.1:6813/3687895811] exists,up 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:20:47.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:20:47.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:20:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:20:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:20:47.710 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:20:47.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:20:47.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:47.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:47.898 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:47.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803788 2026-02-17T12:20:47.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803788 2026-02-17T12:20:47.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788' 2026-02-17T12:20:47.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:47.960 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:48.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757251 2026-02-17T12:20:48.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757251 2026-02-17T12:20:48.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-163208757251' 2026-02-17T12:20:48.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:48.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:48.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411337 2026-02-17T12:20:48.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411337 2026-02-17T12:20:48.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-163208757251 2-77309411337' 2026-02-17T12:20:48.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:48.086 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:20:48.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149703 2026-02-17T12:20:48.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149703 2026-02-17T12:20:48.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-163208757251 2-77309411337 3-111669149703' 2026-02-17T12:20:48.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:48.149 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803788 2026-02-17T12:20:48.149 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:48.149 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:48.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803788 2026-02-17T12:20:48.150 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:48.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803788 2026-02-17T12:20:48.151 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-17T12:20:48.152 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803788 2026-02-17T12:20:48.152 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-17T12:20:48.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:49.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:49.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:49.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-17T12:20:49.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:50.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:20:50.532 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:50.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803788 2026-02-17T12:20:50.720 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:50.720 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-163208757251 2026-02-17T12:20:50.720 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:50.721 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:50.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-163208757251 2026-02-17T12:20:50.722 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:50.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757251 2026-02-17T12:20:50.723 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 163208757251' 2026-02-17T12:20:50.723 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 163208757251 2026-02-17T12:20:50.724 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:50.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-17T12:20:50.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:50.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411337 2026-02-17T12:20:50.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:50.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:50.916 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411337 2026-02-17T12:20:50.916 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:50.916 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411337 2026-02-17T12:20:50.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-17T12:20:50.917 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 77309411337 2026-02-17T12:20:50.917 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:51.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-17T12:20:51.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:51.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149703 2026-02-17T12:20:51.109 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:51.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:20:51.110 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149703 2026-02-17T12:20:51.110 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:51.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149703 2026-02-17T12:20:51.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-17T12:20:51.111 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149703 2026-02-17T12:20:51.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:20:51.300 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149704 -lt 111669149703 2026-02-17T12:20:51.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:20:51.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:51.301 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:20:51.563 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:20:51.564 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:20:51.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:20:51.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:20:51.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:51.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:52.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:20:52.006 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:20:52.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:20:52.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:20:52.007 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:20:52.273 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:20:52.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-10 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:20:52.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-1-10 td/test-erasure-eio/COPY 2026-02-17T12:20:52.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:20:52.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:20:52.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 1 + 1 2026-02-17T12:20:52.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=2 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-1-10 td/test-erasure-eio 2 10 set 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-10 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=2 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=10 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:20:52.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-10 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-10 2026-02-17T12:20:52.310 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:20:52.500 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=2 2026-02-17T12:20:52.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:20:52.779 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:20:52.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:20:52.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 10 = 0 ']' 2026-02-17T12:20:52.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:10 bytes copied, 3.8601e-05 s, 259 kB/s 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:20:52.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:20:52.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:20:52.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:20:52.791 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:20:54.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-17T12:20:54.098 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-92877-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:20:55.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:55.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:20:55.204 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:20:55.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:20:55.206 INFO:tasks.workunit.client.0.trial044.stderr:start osd.2 2026-02-17T12:20:55.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:20:55.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:20:55.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:20:55.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:20:55.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:20:55.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:20:55.218 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:55.216+0000 7f35cecce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:55.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:55.218+0000 7f35cecce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:55.220 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:55.219+0000 7f35cecce780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:55.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:55.394 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:20:55.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:55.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:55.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:56.552 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:56.551+0000 7f35cecce780 -1 Falling back to public interface 2026-02-17T12:20:56.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:56.583 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:20:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:56.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:56.771 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:57.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:57.773 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:20:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:57.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:57.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:58.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:58.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:58.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:58.963 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:20:58.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:58.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:59.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:59.266 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:20:59.265+0000 7f35cecce780 -1 osd.2 41 log_to_monitors true 2026-02-17T12:21:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:00.156 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:00.156 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:00.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 up in weight 1 up_from 47 up_thru 35 down_at 44 last_clean_interval [18,41) [v2:127.0.0.1:6818/1594069024,v1:127.0.0.1:6819/1594069024] [v2:127.0.0.1:6820/1594069024,v1:127.0.0.1:6821/1594069024] exists,up b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:00.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:00.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:00.431 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:00.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:00.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803793 2026-02-17T12:21:00.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803793 2026-02-17T12:21:00.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793' 2026-02-17T12:21:00.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:00.743 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757255 2026-02-17T12:21:00.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757255 2026-02-17T12:21:00.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-163208757255' 2026-02-17T12:21:00.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.744 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:00.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=201863462915 2026-02-17T12:21:00.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 201863462915 2026-02-17T12:21:00.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-163208757255 2-201863462915' 2026-02-17T12:21:00.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.807 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:00.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149708 2026-02-17T12:21:00.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149708 2026-02-17T12:21:00.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-163208757255 2-201863462915 3-111669149708' 2026-02-17T12:21:00.869 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:00.870 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803793 2026-02-17T12:21:00.870 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:00.870 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:00.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803793 2026-02-17T12:21:00.871 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:00.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803793 2026-02-17T12:21:00.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-17T12:21:00.872 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803793 2026-02-17T12:21:00.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:02.061 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:02.062 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:02.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-17T12:21:02.251 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:02.252 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-163208757255 2026-02-17T12:21:02.252 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:02.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:02.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-163208757255 2026-02-17T12:21:02.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:02.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757255 2026-02-17T12:21:02.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 163208757255' 2026-02-17T12:21:02.254 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 163208757255 2026-02-17T12:21:02.255 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:02.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757256 -lt 163208757255 2026-02-17T12:21:02.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:02.444 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-201863462915 2026-02-17T12:21:02.444 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:02.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:02.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-201863462915 2026-02-17T12:21:02.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:02.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=201863462915 2026-02-17T12:21:02.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 201863462915' 2026-02-17T12:21:02.446 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 201863462915 2026-02-17T12:21:02.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:02.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 201863462915 -lt 201863462915 2026-02-17T12:21:02.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:02.638 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149708 2026-02-17T12:21:02.638 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:02.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:21:02.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149708 2026-02-17T12:21:02.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:02.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149708 2026-02-17T12:21:02.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149708' 2026-02-17T12:21:02.641 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149708 2026-02-17T12:21:02.641 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:21:02.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-17T12:21:02.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:21:02.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:02.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:03.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:21:03.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:21:03.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:21:03.279 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:21:03.280 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:03.280 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:03.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:21:03.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:21:03.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:21:03.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:21:03.532 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:21:03.797 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:21:03.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-10 fail 2026-02-17T12:21:03.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:21:03.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:21:03.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-10 2026-02-17T12:21:03.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:21:03.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:21:03.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-1-10 td/test-erasure-eio/COPY 2026-02-17T12:21:03.827 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-1-10: (5) Input/output error 2026-02-17T12:21:03.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:21:03.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:21:03.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:324: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=1 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=0 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=set 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:21:03.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-1-0 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-1-0 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:21:03.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:03.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:21:03.834 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-1-0 td/test-erasure-eio/ORIGINAL 2026-02-17T12:21:03.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-1-0 td/test-erasure-eio 1 0 set 2026-02-17T12:21:03.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-0 2026-02-17T12:21:03.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:21:03.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:21:03.853 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=0 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:21:03.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:03.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:21:03.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-0 2026-02-17T12:21:03.855 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:03.855 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:21:04.041 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:21:04.041 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:04.041 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:21:04.042 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:21:04.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:21:04.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:21:04.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:21:04.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:21:04.303 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:21:04.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:21:04.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 0 = 0 ']' 2026-02-17T12:21:04.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:21:04.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:21:04.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:21:05.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:05.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:06.724 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:21:06.725 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:21:06.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:21:06.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:21:06.727 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:21:06.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:21:06.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:21:06.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:21:06.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:21:06.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:21:06.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:21:06.743 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:06.742+0000 7f7f0469d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:06.744 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:06.743+0000 7f7f0469d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:06.745 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:06.744+0000 7f7f0469d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:06.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:21:06.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:21:06.917 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:06.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:07.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:08.078 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:08.076+0000 7f7f0469d780 -1 Falling back to public interface 2026-02-17T12:21:08.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:08.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:08.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:21:08.106 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:08.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:08.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:08.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:09.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:09.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:09.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:21:09.295 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:09.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:09.295 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:09.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:10.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:10.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:10.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:21:10.486 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:21:10.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:10.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:10.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:10.798 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:10.797+0000 7f7f0469d780 -1 osd.1 51 log_to_monitors true 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:11.675 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 57 up_thru 57 down_at 54 last_clean_interval [38,51) [v2:127.0.0.1:6810/1615060092,v1:127.0.0.1:6811/1615060092] [v2:127.0.0.1:6812/1615060092,v1:127.0.0.1:6813/1615060092] exists,up 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:11.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:11.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:11.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:11.864 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:11.865 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:11.865 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:11.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:11.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:11.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:11.951 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:12.136 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.137 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:12.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803797 2026-02-17T12:21:12.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803797 2026-02-17T12:21:12.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797' 2026-02-17T12:21:12.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:12.263 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135875 2026-02-17T12:21:12.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135875 2026-02-17T12:21:12.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-244813135875' 2026-02-17T12:21:12.264 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.264 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:12.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=201863462919 2026-02-17T12:21:12.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 201863462919 2026-02-17T12:21:12.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-244813135875 2-201863462919' 2026-02-17T12:21:12.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.324 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:12.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149712 2026-02-17T12:21:12.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149712 2026-02-17T12:21:12.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803797 1-244813135875 2-201863462919 3-111669149712' 2026-02-17T12:21:12.386 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:12.387 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803797 2026-02-17T12:21:12.387 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:12.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:12.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803797 2026-02-17T12:21:12.389 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:12.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803797 2026-02-17T12:21:12.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-17T12:21:12.390 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803797 2026-02-17T12:21:12.391 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:12.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-17T12:21:12.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:13.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:13.586 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:13.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-17T12:21:13.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:13.778 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-244813135875 2026-02-17T12:21:13.779 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:13.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:13.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-244813135875 2026-02-17T12:21:13.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:13.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135875 2026-02-17T12:21:13.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 244813135875' 2026-02-17T12:21:13.781 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 244813135875 2026-02-17T12:21:13.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:13.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135875 -lt 244813135875 2026-02-17T12:21:13.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:13.970 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-201863462919 2026-02-17T12:21:13.971 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:13.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:13.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-201863462919 2026-02-17T12:21:13.972 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:13.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=201863462919 2026-02-17T12:21:13.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 201863462919' 2026-02-17T12:21:13.973 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 201863462919 2026-02-17T12:21:13.973 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:14.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-17T12:21:14.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:14.168 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149712 2026-02-17T12:21:14.168 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:14.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:21:14.170 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149712 2026-02-17T12:21:14.170 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:14.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149712 2026-02-17T12:21:14.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149712' 2026-02-17T12:21:14.171 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149712 2026-02-17T12:21:14.171 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:21:14.358 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149712 -lt 111669149712 2026-02-17T12:21:14.359 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:21:14.359 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:14.359 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:14.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:21:14.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:21:14.614 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:21:14.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:21:14.801 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:21:14.802 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:14.802 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:15.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:21:15.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:21:15.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:21:15.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:21:15.062 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:21:15.323 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:21:15.331 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-0 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:21:15.332 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-1-0 td/test-erasure-eio/COPY 2026-02-17T12:21:15.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:21:15.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:21:15.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 1 + 1 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=2 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-1-0 td/test-erasure-eio 2 0 set 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-0 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=2 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=0 2026-02-17T12:21:15.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=set 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-0 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-0 2026-02-17T12:21:15.357 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:21:15.547 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=2 2026-02-17T12:21:15.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:21:15.828 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:21:15.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' set = add ']' 2026-02-17T12:21:15.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: '[' 0 = 0 ']' 2026-02-17T12:21:15.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:21:15.839 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:21:15.840 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:21:17.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-17T12:21:17.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-92877-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:18.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-17T12:21:18.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:21:18.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:21:18.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:21:18.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:21:18.238 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:21:18.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:21:18.239 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:21:18.239 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:21:18.239 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:21:18.239 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:21:18.241 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:start osd.2 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:21:18.242 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:21:18.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:21:18.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:21:18.255 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:18.254+0000 7f39ede07780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:18.257 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:18.255+0000 7f39ede07780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:18.257 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:18.256+0000 7f39ede07780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:21:18.428 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:18.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:21:18.429 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:21:18.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:18.429 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:18.616 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:19.596 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:19.594+0000 7f39ede07780 -1 Falling back to public interface 2026-02-17T12:21:19.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:19.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:19.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:21:19.617 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:19.617 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:19.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:19.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:20.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:20.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:20.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:21:20.809 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:20.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:20.810 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:21.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:22.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:22.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:22.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:21:22.002 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:21:22.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:22.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:22.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:22.304 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:22.301+0000 7f39ede07780 -1 osd.2 62 log_to_monitors true 2026-02-17T12:21:23.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:23.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:23.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:23.196 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:23.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:23.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:23.390 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 up in weight 1 up_from 66 up_thru 54 down_at 63 last_clean_interval [47,62) [v2:127.0.0.1:6818/2351498646,v1:127.0.0.1:6819/2351498646] [v2:127.0.0.1:6820/2351498646,v1:127.0.0.1:6821/2351498646] exists,up b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:21:23.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:23.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:23.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:23.391 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:23.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:23.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:23.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:23.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:23.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:23.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:23.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:23.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:23.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:23.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:23.666 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:23.666 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:23.666 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:23.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:23.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:23.667 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:23.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803801 2026-02-17T12:21:23.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803801 2026-02-17T12:21:23.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803801' 2026-02-17T12:21:23.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:23.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:23.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135879 2026-02-17T12:21:23.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135879 2026-02-17T12:21:23.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803801 1-244813135879' 2026-02-17T12:21:23.790 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:23.790 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:23.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841539 2026-02-17T12:21:23.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841539 2026-02-17T12:21:23.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803801 1-244813135879 2-283467841539' 2026-02-17T12:21:23.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:23.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:23.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149716 2026-02-17T12:21:23.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149716 2026-02-17T12:21:23.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803801 1-244813135879 2-283467841539 3-111669149716' 2026-02-17T12:21:23.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:23.912 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803801 2026-02-17T12:21:23.912 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:23.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:23.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803801 2026-02-17T12:21:23.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:23.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803801 2026-02-17T12:21:23.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803801' 2026-02-17T12:21:23.915 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803801 2026-02-17T12:21:23.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:24.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803799 -lt 25769803801 2026-02-17T12:21:24.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:25.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:25.105 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:25.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803799 -lt 25769803801 2026-02-17T12:21:25.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:26.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:21:26.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:26.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803802 -lt 25769803801 2026-02-17T12:21:26.489 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.489 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-244813135879 2026-02-17T12:21:26.490 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:26.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-244813135879 2026-02-17T12:21:26.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135879 2026-02-17T12:21:26.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 244813135879' 2026-02-17T12:21:26.493 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 244813135879 2026-02-17T12:21:26.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:26.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135879 -lt 244813135879 2026-02-17T12:21:26.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841539 2026-02-17T12:21:26.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:26.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841539 2026-02-17T12:21:26.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841539 2026-02-17T12:21:26.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841539' 2026-02-17T12:21:26.689 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 283467841539 2026-02-17T12:21:26.690 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:26.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841539 -lt 283467841539 2026-02-17T12:21:26.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149716 2026-02-17T12:21:26.877 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:21:26.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149716 2026-02-17T12:21:26.879 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149716 2026-02-17T12:21:26.880 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149716' 2026-02-17T12:21:26.880 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149716 2026-02-17T12:21:26.880 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:21:27.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149717 -lt 111669149716 2026-02-17T12:21:27.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:21:27.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:27.071 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:27.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:21:27.336 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:21:27.523 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:21:27.524 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:21:27.524 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:27.524 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:27.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:21:27.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:21:27.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:21:27.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:21:27.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:21:28.052 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-0 fail 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-0 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:21:28.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:21:28.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-1-0 td/test-erasure-eio/COPY 2026-02-17T12:21:28.082 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-1-0: (5) Input/output error 2026-02-17T12:21:28.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:21:28.083 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:21:28.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-17T12:21:28.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:223: rados_get_data_bad_size: shift 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: rados_get_data_bad_size: local shard_id=1 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:225: rados_get_data_bad_size: shift 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: rados_get_data_bad_size: local bytes=256 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: rados_get_data_bad_size: shift 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: rados_get_data_bad_size: local mode=add 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:230: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:231: rados_get_data_bad_size: local objname=obj-size-92877-1-256 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:21:28.087 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-92877-1-256 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:21:28.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:21:28.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-92877-1-256 td/test-erasure-eio/ORIGINAL 2026-02-17T12:21:28.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: set_size obj-size-92877-1-256 td/test-erasure-eio 1 256 add 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-256 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=1 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=256 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=add 2026-02-17T12:21:28.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:21:28.108 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:28.108 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:21:28.108 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-256 2026-02-17T12:21:28.108 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:28.108 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:21:28.297 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=1 2026-02-17T12:21:28.298 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:21:28.558 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:21:28.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' add = add ']' 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:21:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:21:28.571 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:21:28.571 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:21:28.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:21:28.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:21:28.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:21:29.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:21:29.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:29.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:21:29.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:21:29.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:21:29.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:21:29.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:29.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:21:30.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:21:30.692 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:21:30.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:21:30.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:21:30.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:21:30.710 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:30.709+0000 7fc4204d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:30.711 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:30.710+0000 7fc4204d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:30.712 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:30.711+0000 7fc4204d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:30.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:21:30.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:30.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:31.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:32.059 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:32.058+0000 7fc4204d2780 -1 Falling back to public interface 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:32.071 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:32.259 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:33.261 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:33.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:34.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:34.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:34.782 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:34.780+0000 7fc4204d2780 -1 osd.1 71 log_to_monitors true 2026-02-17T12:21:35.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:35.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:35.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:35.639 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:35.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:35.640 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:35.829 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 75 up_thru 75 down_at 72 last_clean_interval [57,71) [v2:127.0.0.1:6810/3307261349,v1:127.0.0.1:6811/3307261349] [v2:127.0.0.1:6812/3307261349,v1:127.0.0.1:6813/3307261349] exists,up 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:21:35.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:35.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:35.830 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:35.831 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:35.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:35.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:35.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:35.915 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:36.106 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:36.106 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:36.106 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:36.107 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:36.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:36.107 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.107 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:36.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803806 2026-02-17T12:21:36.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803806 2026-02-17T12:21:36.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806' 2026-02-17T12:21:36.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.168 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:36.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547203 2026-02-17T12:21:36.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547203 2026-02-17T12:21:36.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-322122547203' 2026-02-17T12:21:36.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.231 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:36.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841543 2026-02-17T12:21:36.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841543 2026-02-17T12:21:36.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-322122547203 2-283467841543' 2026-02-17T12:21:36.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.292 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:36.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149721 2026-02-17T12:21:36.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149721 2026-02-17T12:21:36.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803806 1-322122547203 2-283467841543 3-111669149721' 2026-02-17T12:21:36.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:36.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803806 2026-02-17T12:21:36.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:36.354 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:36.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803806 2026-02-17T12:21:36.355 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803806 2026-02-17T12:21:36.356 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803806' 2026-02-17T12:21:36.356 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803806 2026-02-17T12:21:36.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:36.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803804 -lt 25769803806 2026-02-17T12:21:36.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:37.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:37.546 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:37.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803806 -lt 25769803806 2026-02-17T12:21:37.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:37.733 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547203 2026-02-17T12:21:37.733 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:37.734 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:37.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547203 2026-02-17T12:21:37.734 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:37.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547203 2026-02-17T12:21:37.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547203' 2026-02-17T12:21:37.736 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 322122547203 2026-02-17T12:21:37.736 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:37.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547203 -lt 322122547203 2026-02-17T12:21:37.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:37.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841543 2026-02-17T12:21:37.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:37.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:37.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841543 2026-02-17T12:21:37.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:37.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841543 2026-02-17T12:21:37.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841543' 2026-02-17T12:21:37.928 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 283467841543 2026-02-17T12:21:37.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:38.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841544 -lt 283467841543 2026-02-17T12:21:38.116 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:38.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149721 2026-02-17T12:21:38.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:38.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:21:38.118 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149721 2026-02-17T12:21:38.118 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:38.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149721 2026-02-17T12:21:38.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149721' 2026-02-17T12:21:38.119 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149721 2026-02-17T12:21:38.119 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:21:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149721 -lt 111669149721 2026-02-17T12:21:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:21:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:21:38.560 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:21:38.561 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:21:38.561 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:21:38.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:21:38.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:21:38.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:38.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:38.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:21:38.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:21:38.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:21:38.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-17T12:21:38.999 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:21:38.999 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:256 bytes copied, 2.2121e-05 s, 11.6 MB/s 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:21:39.000 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:21:39.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:21:40.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:21:40.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:40.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:41.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:21:41.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:21:41.404 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:21:41.405 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:21:41.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:21:41.407 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:21:41.408 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:21:41.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:21:41.418 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:41.417+0000 7fbb55ab1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:41.420 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:41.418+0000 7fbb55ab1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:41.421 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:41.419+0000 7fbb55ab1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:41.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:21:41.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:41.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:42.763 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:42.761+0000 7fbb55ab1780 -1 Falling back to public interface 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:42.793 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:42.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:43.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:43.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:43.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:21:43.987 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:43.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:43.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:44.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:45.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:45.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:45.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:21:45.176 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:21:45.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:45.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:45.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:45.446 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:45.445+0000 7fbb55ab1780 -1 osd.1 77 log_to_monitors true 2026-02-17T12:21:46.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:46.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:46.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:46.364 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:46.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:46.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:21:46.550 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 81 up_thru 81 down_at 78 last_clean_interval [75,77) [v2:127.0.0.1:6810/969516920,v1:127.0.0.1:6811/969516920] [v2:127.0.0.1:6812/969516920,v1:127.0.0.1:6813/969516920] exists,up 48af8c38-1c26-42fa-a196-d2d8888edbc3 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:46.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:46.552 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:46.552 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:46.552 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:46.552 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:46.552 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:46.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:46.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:46.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:46.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:46.639 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:46.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:46.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:46.827 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:46.828 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:46.828 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:46.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:46.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:46.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:46.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803810 2026-02-17T12:21:46.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803810 2026-02-17T12:21:46.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810' 2026-02-17T12:21:46.889 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:46.889 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:46.951 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350979 2026-02-17T12:21:46.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350979 2026-02-17T12:21:46.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-347892350979' 2026-02-17T12:21:46.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:46.952 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:47.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841547 2026-02-17T12:21:47.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841547 2026-02-17T12:21:47.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-347892350979 2-283467841547' 2026-02-17T12:21:47.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:47.014 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:47.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149725 2026-02-17T12:21:47.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149725 2026-02-17T12:21:47.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803810 1-347892350979 2-283467841547 3-111669149725' 2026-02-17T12:21:47.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:47.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803810 2026-02-17T12:21:47.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:47.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:47.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803810 2026-02-17T12:21:47.082 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:47.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803810 2026-02-17T12:21:47.082 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803810' 2026-02-17T12:21:47.083 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803810 2026-02-17T12:21:47.083 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803808 -lt 25769803810 2026-02-17T12:21:47.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:48.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:48.274 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:48.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803810 -lt 25769803810 2026-02-17T12:21:48.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:48.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-347892350979 2026-02-17T12:21:48.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:48.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:48.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-347892350979 2026-02-17T12:21:48.471 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:48.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350979 2026-02-17T12:21:48.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 347892350979' 2026-02-17T12:21:48.471 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 347892350979 2026-02-17T12:21:48.472 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:48.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350979 -lt 347892350979 2026-02-17T12:21:48.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:48.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841547 2026-02-17T12:21:48.661 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:48.661 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:48.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841547 2026-02-17T12:21:48.662 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:48.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841547 2026-02-17T12:21:48.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841547' 2026-02-17T12:21:48.663 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 283467841547 2026-02-17T12:21:48.664 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:48.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841548 -lt 283467841547 2026-02-17T12:21:48.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:48.855 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149725 2026-02-17T12:21:48.856 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:48.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:21:48.857 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149725 2026-02-17T12:21:48.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:48.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149725 2026-02-17T12:21:48.859 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149725' 2026-02-17T12:21:48.859 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149725 2026-02-17T12:21:48.859 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:21:49.048 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149725 -lt 111669149725 2026-02-17T12:21:49.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:21:49.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:49.049 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:21:49.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:21:49.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:21:49.496 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:21:49.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:21:49.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:21:49.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:21:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:21:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:21:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:21:49.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:21:50.054 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-256 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:21:50.064 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:21:50.065 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-92877-1-256 td/test-erasure-eio/COPY 2026-02-17T12:21:50.084 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:21:50.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:21:50.086 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: expr 1 + 1 2026-02-17T12:21:50.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: shard_id=2 2026-02-17T12:21:50.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: set_size obj-size-92877-1-256 td/test-erasure-eio 2 256 add 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: set_size: local objname=obj-size-92877-1-256 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:193: set_size: shift 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: set_size: local dir=td/test-erasure-eio 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:195: set_size: shift 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:196: set_size: local shard_id=2 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:197: set_size: shift 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:198: set_size: local bytes=256 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:199: set_size: shift 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:200: set_size: local mode=add 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local poolname=pool-jerasure 2026-02-17T12:21:50.089 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: get_osds pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:50.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:21:50.090 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-size-92877-1-256 2026-02-17T12:21:50.090 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-size-92877-1-256 2026-02-17T12:21:50.090 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: initial_osds=('3' '1' '2') 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: local -a initial_osds 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local osd_id=2 2026-02-17T12:21:50.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: ceph osd set noout 2026-02-17T12:21:50.557 INFO:tasks.workunit.client.0.trial044.stderr:noout is set 2026-02-17T12:21:50.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: '[' add = add ']' 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:21:50.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:21:50.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:21:51.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:21:51.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:51.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:21:51.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:21:51.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:21:51.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:21:51.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-17T12:21:51.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-92877-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:21:52.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-17T12:21:52.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:21:52.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:21:52.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:21:52.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:21:52.690 INFO:tasks.workunit.client.0.trial044.stderr:start osd.2 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:21:52.692 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:21:52.703 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:52.702+0000 7f362ba3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:52.704 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:52.703+0000 7f362ba3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:52.705 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:52.704+0000 7f362ba3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:21:52.878 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:52.879 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:53.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:54.061 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:54.059+0000 7f362ba3f780 -1 Falling back to public interface 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:54.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:54.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:55.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:55.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:56.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:21:56.446 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:21:56.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:56.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:56.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:21:56.787 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:21:56.785+0000 7f362ba3f780 -1 osd.2 85 log_to_monitors true 2026-02-17T12:21:57.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:21:57.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:21:57.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:21:57.635 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:21:57.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:21:57.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:21:57.826 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 up in weight 1 up_from 90 up_thru 78 down_at 87 last_clean_interval [66,85) [v2:127.0.0.1:6818/3689418052,v1:127.0.0.1:6819/3689418052] [v2:127.0.0.1:6820/3689418052,v1:127.0.0.1:6821/3689418052] exists,up b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:21:57.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:21:57.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:21:57.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:21:57.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:21:57.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:21:57.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:21:57.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:21:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:21:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:21:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:21:57.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:57.913 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:58.102 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:58.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803814 2026-02-17T12:21:58.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803814 2026-02-17T12:21:58.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803814' 2026-02-17T12:21:58.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:58.164 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:58.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350983 2026-02-17T12:21:58.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350983 2026-02-17T12:21:58.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803814 1-347892350983' 2026-02-17T12:21:58.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:58.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:58.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056643 2026-02-17T12:21:58.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056643 2026-02-17T12:21:58.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803814 1-347892350983 2-386547056643' 2026-02-17T12:21:58.289 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:58.289 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:21:58.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149729 2026-02-17T12:21:58.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149729 2026-02-17T12:21:58.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803814 1-347892350983 2-386547056643 3-111669149729' 2026-02-17T12:21:58.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:58.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803814 2026-02-17T12:21:58.353 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:58.353 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:58.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803814 2026-02-17T12:21:58.354 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:58.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803814 2026-02-17T12:21:58.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803814' 2026-02-17T12:21:58.355 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803814 2026-02-17T12:21:58.356 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:58.543 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803812 -lt 25769803814 2026-02-17T12:21:58.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:59.544 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:59.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:59.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803815 -lt 25769803814 2026-02-17T12:21:59.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:59.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-347892350983 2026-02-17T12:21:59.737 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:59.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:59.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-347892350983 2026-02-17T12:21:59.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:59.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350983 2026-02-17T12:21:59.739 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 347892350983' 2026-02-17T12:21:59.739 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 347892350983 2026-02-17T12:21:59.739 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:59.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350983 -lt 347892350983 2026-02-17T12:21:59.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:59.934 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-386547056643 2026-02-17T12:21:59.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:59.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:59.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-386547056643 2026-02-17T12:21:59.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:59.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056643 2026-02-17T12:21:59.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 386547056643' 2026-02-17T12:21:59.937 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 386547056643 2026-02-17T12:21:59.937 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-17T12:22:00.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:00.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149729 2026-02-17T12:22:00.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:00.126 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:22:00.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149729 2026-02-17T12:22:00.127 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:00.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149729 2026-02-17T12:22:00.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149729' 2026-02-17T12:22:00.128 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149729 2026-02-17T12:22:00.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:22:00.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149730 -lt 111669149729 2026-02-17T12:22:00.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:22:00.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:00.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:22:00.576 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:22:00.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:22:00.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:22:00.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:00.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:01.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:22:01.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:22:01.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:22:01.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-17T12:22:01.021 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records in 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:1+0 records out 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:256 bytes copied, 1.8054e-05 s, 14.2 MB/s 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:22:01.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:22:01.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:22:01.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:22:01.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:22:02.328 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-17T12:22:02.329 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-92877-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:22:03.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:03.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:03.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:22:03.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:22:03.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:22:03.422 INFO:tasks.workunit.client.0.trial044.stderr:start osd.2 2026-02-17T12:22:03.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:22:03.423 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:22:03.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:22:03.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:22:03.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:22:03.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:22:03.435 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:03.433+0000 7fc2d6022780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:03.436 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:03.434+0000 7fc2d6022780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:03.437 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:03.435+0000 7fc2d6022780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:22:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:22:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:22:03.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:03.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:03.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:04.777 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:04.774+0000 7fc2d6022780 -1 Falling back to public interface 2026-02-17T12:22:04.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:04.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:04.797 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:22:04.798 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:22:04.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:04.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:04.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:05.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:05.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:05.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:22:05.995 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:22:05.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:05.995 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:06.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:07.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:07.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:07.489 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:07.486+0000 7fc2d6022780 -1 osd.2 91 log_to_monitors true 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:08.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:08.564 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 up in weight 1 up_from 95 up_thru 78 down_at 92 last_clean_interval [90,91) [v2:127.0.0.1:6818/1331041823,v1:127.0.0.1:6819/1331041823] [v2:127.0.0.1:6820/1331041823,v1:127.0.0.1:6821/1331041823] exists,up b2ab1ec0-094b-4965-b566-a37328a0bf92 2026-02-17T12:22:08.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:22:08.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:22:08.564 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:22:08.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:08.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.838 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:08.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803818 2026-02-17T12:22:08.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803818 2026-02-17T12:22:08.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803818' 2026-02-17T12:22:08.902 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.902 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:08.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350987 2026-02-17T12:22:08.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350987 2026-02-17T12:22:08.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803818 1-347892350987' 2026-02-17T12:22:08.966 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:09.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893123 2026-02-17T12:22:09.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893123 2026-02-17T12:22:09.026 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803818 1-347892350987 2-408021893123' 2026-02-17T12:22:09.027 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:09.027 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:22:09.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149734 2026-02-17T12:22:09.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149734 2026-02-17T12:22:09.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803818 1-347892350987 2-408021893123 3-111669149734' 2026-02-17T12:22:09.091 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:09.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803818 2026-02-17T12:22:09.092 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:09.093 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:09.093 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803818 2026-02-17T12:22:09.094 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:09.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803818 2026-02-17T12:22:09.094 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803818' 2026-02-17T12:22:09.095 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803818 2026-02-17T12:22:09.095 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:09.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803816 -lt 25769803818 2026-02-17T12:22:09.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:10.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:10.285 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:10.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803819 -lt 25769803818 2026-02-17T12:22:10.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:10.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-347892350987 2026-02-17T12:22:10.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:10.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:10.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-347892350987 2026-02-17T12:22:10.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:10.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350987 2026-02-17T12:22:10.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 347892350987' 2026-02-17T12:22:10.478 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 347892350987 2026-02-17T12:22:10.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350987 -lt 347892350987 2026-02-17T12:22:10.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:10.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-408021893123 2026-02-17T12:22:10.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:10.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:10.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-408021893123 2026-02-17T12:22:10.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893123 2026-02-17T12:22:10.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 408021893123' 2026-02-17T12:22:10.672 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 408021893123 2026-02-17T12:22:10.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:10.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-17T12:22:10.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:10.861 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149734 2026-02-17T12:22:10.861 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:10.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:22:10.862 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149734 2026-02-17T12:22:10.862 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149734 2026-02-17T12:22:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149734' 2026-02-17T12:22:10.863 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 111669149734 2026-02-17T12:22:10.863 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:22:11.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149734 -lt 111669149734 2026-02-17T12:22:11.052 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:22:11.053 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:11.053 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:11.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:22:11.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:22:11.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:22:11.496 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:22:11.497 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:22:11.498 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:11.498 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:11.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:22:11.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:22:11.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:22:11.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:217: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-17T12:22:11.752 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: ceph osd unset noout 2026-02-17T12:22:12.045 INFO:tasks.workunit.client.0.trial044.stderr:noout is unset 2026-02-17T12:22:12.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:243: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-92877-1-256 fail 2026-02-17T12:22:12.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:22:12.057 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:22:12.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-92877-1-256 2026-02-17T12:22:12.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:22:12.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:22:12.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-92877-1-256 td/test-erasure-eio/COPY 2026-02-17T12:22:12.077 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-size-92877-1-256: (5) Input/output error 2026-02-17T12:22:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:22:12.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:244: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:22:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:22:12.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:22:12.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:22:12.297 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:22:12.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:22:12.503 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:22:12.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:22:12.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:22:12.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:22:12.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:22:12.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:22:12.512 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:22:12.512 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:22:12.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:22:12.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:22:12.620 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:22:12.620 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:22:12.621 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:22:12.622 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:22:12.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:22:12.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:22:12.623 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:22:12.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:12.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:22:12.625 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:22:12.625 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:12.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:22:12.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:22:12.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:22:12.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:22:12.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:22:12.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:22:12.647 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:22:12.647 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:22:12.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:22:12.648 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:22:12.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:22:12.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:22:12.650 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:22:12.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:12.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:22:12.651 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:22:12.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:12.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:22:12.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:22:12.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:22:12.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:22:12.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:22:12.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:22:12.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:22:12.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:22:12.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:22:12.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:22:12.658 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:22:12.659 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:22:12.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:22:12.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.679 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.680 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:12.680 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:22:12.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:22:12.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:22:12.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:22:12.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:22:12.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:22:12.706 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:22:12.707 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:22:12.707 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:12.707 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:22:12.708 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:22:12.752 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:12.753 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:12.754 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:12.754 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.754 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.754 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:12.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:22:12.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:22:12.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:22:12.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:22:12.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:12.899 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:12.900 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:12.900 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:12.900 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:12.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:12.900 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:22:12.901 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:22:12.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:22:12.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:22:13.013 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:22:13.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:22:14.023 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:22:14.063 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:22:14.067 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:22:14.068 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:22:12.696+0000 7f33a9823d80 0 load: jerasure load: lrc 2026-02-17T12:22:14.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-17T12:22:14.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:255: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-17T12:22:14.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:256: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-17T12:22:14.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:22:14.070 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:22:14.070 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:22:14.070 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:14.075 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:14.076 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:14.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:22:14.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:22:14.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:22:14.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:22:14.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e2d7cf53-a24f-40e3-99f8-7295091f9c13 2026-02-17T12:22:14.078 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 e2d7cf53-a24f-40e3-99f8-7295091f9c13 2026-02-17T12:22:14.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 e2d7cf53-a24f-40e3-99f8-7295091f9c13' 2026-02-17T12:22:14.079 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:22:14.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB2XZRpcl8lBRAAO8phY//Eaq872cloDbg77A== 2026-02-17T12:22:14.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB2XZRpcl8lBRAAO8phY//Eaq872cloDbg77A=="}' 2026-02-17T12:22:14.088 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e2d7cf53-a24f-40e3-99f8-7295091f9c13 -i td/test-erasure-eio/0/new.json 2026-02-17T12:22:14.192 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:14.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:22:14.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB2XZRpcl8lBRAAO8phY//Eaq872cloDbg77A== --osd-uuid e2d7cf53-a24f-40e3-99f8-7295091f9c13 2026-02-17T12:22:14.217 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:14.215+0000 7fc39ba18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:14.218 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:14.217+0000 7fc39ba18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:14.219 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:14.218+0000 7fc39ba18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:14.220 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:14.218+0000 7fc39ba18780 -1 bdev(0x560351137c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:22:14.220 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:14.218+0000 7fc39ba18780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:22:16.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:22:16.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:22:16.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:22:16.596 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:22:16.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:22:16.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:22:16.851 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:22:16.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:22:16.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:22:16.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:22:16.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:22:16.867 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:16.866+0000 7feff42af780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:16.868 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:16.867+0000 7feff42af780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:16.869 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:16.868+0000 7feff42af780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:17.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:22:17.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:22:17.040 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:17.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:17.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:17.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:18.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:18.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:18.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:22:18.229 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:22:18.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:18.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:18.416 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:18.467 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:18.466+0000 7feff42af780 -1 Falling back to public interface 2026-02-17T12:22:19.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:19.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:19.418 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:22:19.418 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:22:19.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:19.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:19.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:20.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:20.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:20.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:22:20.610 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:22:20.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:20.610 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:20.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:21.403 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:21.402+0000 7feff42af780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:22:21.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:21.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:21.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:22:21.804 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:22:21.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:21.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:21.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:22.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:22.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:22.998 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:22:22.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:22:22.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:22.999 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:22:23.192 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2542761309,v1:127.0.0.1:6803/2542761309] [v2:127.0.0.1:6804/2542761309,v1:127.0.0.1:6805/2542761309] exists,up e2d7cf53-a24f-40e3-99f8-7295091f9c13 2026-02-17T12:22:23.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:22:23.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:23.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:23.198 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:23.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:22:23.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:22:23.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:22:23.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:22:23.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=37bfe01d-98ef-45be-84e9-311777249730 2026-02-17T12:22:23.200 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 37bfe01d-98ef-45be-84e9-311777249730 2026-02-17T12:22:23.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 37bfe01d-98ef-45be-84e9-311777249730' 2026-02-17T12:22:23.201 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:22:23.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB/XZRpuzBdDBAA3fB5+HSC1ToGrRiYMH5/9Q== 2026-02-17T12:22:23.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB/XZRpuzBdDBAA3fB5+HSC1ToGrRiYMH5/9Q=="}' 2026-02-17T12:22:23.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 37bfe01d-98ef-45be-84e9-311777249730 -i td/test-erasure-eio/1/new.json 2026-02-17T12:22:23.404 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:22:23.412 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:22:23.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB/XZRpuzBdDBAA3fB5+HSC1ToGrRiYMH5/9Q== --osd-uuid 37bfe01d-98ef-45be-84e9-311777249730 2026-02-17T12:22:23.430 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:23.429+0000 7fdb2cf08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:23.431 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:23.430+0000 7fdb2cf08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:23.432 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:23.431+0000 7fdb2cf08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:23.433 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:23.431+0000 7fdb2cf08780 -1 bdev(0x55e95ecf3c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:22:23.433 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:23.431+0000 7fdb2cf08780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:22:25.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:22:25.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:22:25.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:22:25.549 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:22:25.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:22:25.803 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:22:25.803 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:22:25.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:22:25.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:22:25.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:22:25.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:22:25.818 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:25.817+0000 7fe2b7627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:25.820 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:25.819+0000 7fe2b7627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:25.821 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:25.819+0000 7fe2b7627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:25.991 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:22:25.992 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:25.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:25.992 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:22:26.180 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:27.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:27.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:27.181 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:22:27.182 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:22:27.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:27.182 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:22:27.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:27.407 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:27.405+0000 7fe2b7627780 -1 Falling back to public interface 2026-02-17T12:22:28.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:28.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:28.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:22:28.375 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:22:28.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:28.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:22:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:29.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:29.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:29.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:22:29.569 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:22:29.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:29.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:22:29.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:30.348 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:30.346+0000 7fe2b7627780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:30.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:22:30.957 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1978384934,v1:127.0.0.1:6811/1978384934] [v2:127.0.0.1:6812/1978384934,v1:127.0.0.1:6813/1978384934] exists,up 37bfe01d-98ef-45be-84e9-311777249730 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:30.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:22:30.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:30.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ee9df65f-cef9-4f76-a6e0-46e65d3060f3 2026-02-17T12:22:30.965 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 ee9df65f-cef9-4f76-a6e0-46e65d3060f3' 2026-02-17T12:22:30.966 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 ee9df65f-cef9-4f76-a6e0-46e65d3060f3 2026-02-17T12:22:30.966 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:22:30.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCGXZRpFGrqORAAG3jX5+//REFLgYxFjIdKSA== 2026-02-17T12:22:30.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCGXZRpFGrqORAAG3jX5+//REFLgYxFjIdKSA=="}' 2026-02-17T12:22:30.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ee9df65f-cef9-4f76-a6e0-46e65d3060f3 -i td/test-erasure-eio/2/new.json 2026-02-17T12:22:31.165 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:22:31.173 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:22:31.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGXZRpFGrqORAAG3jX5+//REFLgYxFjIdKSA== --osd-uuid ee9df65f-cef9-4f76-a6e0-46e65d3060f3 2026-02-17T12:22:31.192 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:31.190+0000 7f94637d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:31.193 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:31.191+0000 7f94637d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:31.194 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:31.192+0000 7f94637d0780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:31.194 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:31.193+0000 7f94637d0780 -1 bdev(0x55bab4b6bc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:22:31.194 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:31.193+0000 7f94637d0780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:22:33.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:22:33.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:22:33.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:22:33.319 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:22:33.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:22:33.575 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:22:33.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:22:33.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:22:33.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:22:33.576 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:22:33.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:22:33.591 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:33.590+0000 7fc558a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:33.592 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:33.591+0000 7fc558a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:33.593 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:33.592+0000 7fc558a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:33.772 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:22:33.773 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:33.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:33.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:33.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:34.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:34.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:34.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:22:34.959 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:22:34.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:34.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:35.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:35.175 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:35.173+0000 7fc558a18780 -1 Falling back to public interface 2026-02-17T12:22:36.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:36.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:36.147 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:22:36.147 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:22:36.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:36.148 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:36.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:37.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:37.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:37.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:22:37.337 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:22:37.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:37.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:37.533 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:38.139 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:38.137+0000 7fc558a18780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:22:38.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:38.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:38.535 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:22:38.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:22:38.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:38.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:38.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:39.729 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:22:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:22:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:22:39.920 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1216320412,v1:127.0.0.1:6819/1216320412] [v2:127.0.0.1:6820/1216320412,v1:127.0.0.1:6821/1216320412] exists,up ee9df65f-cef9-4f76-a6e0-46e65d3060f3 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:22:39.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:39.925 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:39.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5320e2ac-bc82-4edb-b4a2-7f77385cf44f 2026-02-17T12:22:39.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 5320e2ac-bc82-4edb-b4a2-7f77385cf44f' 2026-02-17T12:22:39.928 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 5320e2ac-bc82-4edb-b4a2-7f77385cf44f 2026-02-17T12:22:39.928 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:22:39.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCPXZRpExO0NxAAlHnNcTD4j8IqtBd9nlaFow== 2026-02-17T12:22:39.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCPXZRpExO0NxAAlHnNcTD4j8IqtBd9nlaFow=="}' 2026-02-17T12:22:39.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5320e2ac-bc82-4edb-b4a2-7f77385cf44f -i td/test-erasure-eio/3/new.json 2026-02-17T12:22:40.127 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:22:40.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:22:40.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCPXZRpExO0NxAAlHnNcTD4j8IqtBd9nlaFow== --osd-uuid 5320e2ac-bc82-4edb-b4a2-7f77385cf44f 2026-02-17T12:22:40.153 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:40.150+0000 7fceb14d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:40.154 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:40.152+0000 7fceb14d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:40.155 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:40.153+0000 7fceb14d2780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:40.155 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:40.153+0000 7fceb14d2780 -1 bdev(0x563c91b75c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:22:40.155 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:40.153+0000 7fceb14d2780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:22:42.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:22:42.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:22:42.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:22:42.515 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:22:42.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:22:42.781 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:22:42.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:22:42.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:22:42.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:22:42.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:22:42.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:22:42.798 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:42.795+0000 7f15b4411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:42.799 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:42.797+0000 7f15b4411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:42.800 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:42.798+0000 7f15b4411780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:42.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:43.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:44.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:44.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:44.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:22:44.159 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:22:44.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:44.160 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:44.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:44.384 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:44.382+0000 7f15b4411780 -1 Falling back to public interface 2026-02-17T12:22:45.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:45.348 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:45.349 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:22:45.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:22:45.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:45.349 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:45.535 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:46.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:46.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:46.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:22:46.536 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:22:46.536 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:46.537 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:46.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:47.327 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:47.324+0000 7f15b4411780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:22:47.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:47.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:47.729 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:22:47.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:22:47.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:47.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:47.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:22:48.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:22:48.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:22:48.919 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:22:48.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:22:48.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:22:48.920 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:22:49.109 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4010206180,v1:127.0.0.1:6827/4010206180] [v2:127.0.0.1:6828/4010206180,v1:127.0.0.1:6829/4010206180] exists,up 5320e2ac-bc82-4edb-b4a2-7f77385cf44f 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:22:49.111 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:49.112 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:49.112 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:49.112 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:22:49.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:22:49.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:22:49.156 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:22:49.160 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:22:49.162 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:22:18.468+0000 7feff42af780 0 load: jerasure load: lrc 2026-02-17T12:22:49.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-17T12:22:49.163 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:259: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:22:49.164 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:22:49.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:22:49.413 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:22:49.708 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:22:49.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:22:50.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:22:50.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:22:50.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:22:50.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:22:50.718 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:22:50.719 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:22:50.719 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:22:50.719 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:22:50.720 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:22:50.720 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:22:50.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:22:50.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:22:50.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:22:50.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:22:50.808 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:50.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:50.996 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:50.997 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:51.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:22:51.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:22:51.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:22:51.060 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:51.060 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:51.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-17T12:22:51.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-17T12:22:51.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-55834574854' 2026-02-17T12:22:51.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:51.121 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:51.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411333 2026-02-17T12:22:51.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411333 2026-02-17T12:22:51.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-55834574854 2-77309411333' 2026-02-17T12:22:51.184 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:51.184 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:22:51.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T12:22:51.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T12:22:51.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-55834574854 2-77309411333 3-111669149699' 2026-02-17T12:22:51.246 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:51.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:22:51.246 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:51.247 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:51.247 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:22:51.247 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:51.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:22:51.249 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:22:51.249 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:22:51.249 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:51.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:22:51.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:52.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:52.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:52.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:22:52.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:53.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:22:53.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:53.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803785 -lt 25769803784 2026-02-17T12:22:53.826 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:53.827 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-55834574854 2026-02-17T12:22:53.827 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:53.827 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:53.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-55834574854 2026-02-17T12:22:53.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:53.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-17T12:22:53.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-17T12:22:53.829 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 55834574854 2026-02-17T12:22:53.830 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:54.018 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574855 -lt 55834574854 2026-02-17T12:22:54.019 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:54.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411333 2026-02-17T12:22:54.019 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:54.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:54.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411333 2026-02-17T12:22:54.021 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:54.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411333 2026-02-17T12:22:54.022 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-17T12:22:54.022 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411333 2026-02-17T12:22:54.023 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:54.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-17T12:22:54.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:54.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149699 2026-02-17T12:22:54.211 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:54.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:22:54.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149699 2026-02-17T12:22:54.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:54.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T12:22:54.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-17T12:22:54.213 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149699 2026-02-17T12:22:54.214 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:22:54.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T12:22:54.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:22:54.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:54.404 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:54.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:22:54.665 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:22:54.666 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:22:54.851 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:22:54.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:22:54.851 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:22:54.852 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:22:55.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:261: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-17T12:22:55.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-92877-0 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-92877-0 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:22:55.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:22:55.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:22:55.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:22:55.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:22:55.114 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-92877-0 td/test-erasure-eio/ORIGINAL 2026-02-17T12:22:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-0 td/test-erasure-eio 0 2026-02-17T12:22:55.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-0 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:22:55.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:22:55.129 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-0 2026-02-17T12:22:55.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.129 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:22:55.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:22:55.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:22:55.322 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:22:55.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:22:55.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:22:55.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:22:55.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:22:55.323 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:22:55.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:22:55.324 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:22:55.324 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:22:55.324 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:22:55.324 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:22:55.325 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:55.325 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:55.325 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:55.325 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:22:55.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:22:55.372 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:22:55.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:22:55.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:22:55.374 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:22:55.374 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:22:55.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-92877-0 0 2026-02-17T12:22:55.422 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-0 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:22:55.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-92877-0 td/test-erasure-eio/COPY 2026-02-17T12:22:55.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:22:55.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:22:55.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-17T12:22:55.443 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-17T12:22:55.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: shard_id=1 2026-02-17T12:22:55.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-0 td/test-erasure-eio 1 2026-02-17T12:22:55.445 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-0 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:22:55.446 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:22:55.447 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-0 2026-02-17T12:22:55.447 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-0 2026-02-17T12:22:55.447 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:22:55.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:22:55.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:22:55.635 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/1/type 2026-02-17T12:22:55.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:55.637 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:55.638 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:55.638 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:22:55.638 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:22:55.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:22:55.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:55.688 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:22:55.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:22:55.689 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-92877-0 1 2026-02-17T12:22:55.735 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-0 fail 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-0 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:22:55.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-92877-0 td/test-erasure-eio/COPY 2026-02-17T12:22:55.753 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-eio-92877-0: (5) Input/output error 2026-02-17T12:22:55.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:22:55.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:184: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:22:55.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:263: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:22:55.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:22:55.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:22:55.987 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:22:55.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:22:56.197 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:22:56.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:22:56.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:22:56.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:22:56.315 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:22:56.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:22:56.316 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:22:56.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:22:56.318 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:22:56.318 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:22:56.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:56.319 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:22:56.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:22:56.320 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:56.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:22:56.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:22:56.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:22:56.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:22:56.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.331 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.331 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:22:56.333 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:22:56.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:22:56.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:22:56.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:22:56.334 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:22:56.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:22:56.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:22:56.335 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:22:56.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:22:56.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:22:56.337 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:22:56.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:56.337 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:22:56.338 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:22:56.338 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:22:56.339 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:22:56.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:22:56.340 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:22:56.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:22:56.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.341 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.341 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:22:56.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:22:56.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:22:56.342 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:22:56.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:22:56.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.343 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.343 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:22:56.344 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:22:56.346 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:22:56.347 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:22:56.364 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.365 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:56.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:22:56.391 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:22:56.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:22:56.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:22:56.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:22:56.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:22:56.392 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:22:56.394 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:22:56.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:22:56.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:22:56.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:22:56.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:22:56.439 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:22:56.440 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:22:56.441 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:22:56.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:22:56.585 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:22:56.585 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:56.585 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:56.585 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:56.586 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:56.586 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:56.586 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:56.586 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:22:56.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:22:56.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:22:56.603 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:22:56.700 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:22:56.707 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:22:57.710 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:22:57.710 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:22:57.710 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:22:57.710 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:22:57.711 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:57.711 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:57.711 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:22:57.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:22:57.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:22:57.752 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:22:57.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:22:57.757 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:22:56.382+0000 7f9849722d80 0 load: jerasure load: lrc 2026-02-17T12:22:57.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-17T12:22:57.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:267: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-17T12:22:57.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:268: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-17T12:22:57.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:22:57.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:22:57.759 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:22:57.759 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:22:57.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:22:57.764 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:22:57.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:22:57.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:22:57.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:22:57.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:22:57.766 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:22:57.766 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1bc4b594-93b9-4ec8-b077-96fcd892174c 2026-02-17T12:22:57.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1bc4b594-93b9-4ec8-b077-96fcd892174c' 2026-02-17T12:22:57.767 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 1bc4b594-93b9-4ec8-b077-96fcd892174c 2026-02-17T12:22:57.767 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:22:57.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQChXZRpAtwgLhAAr3akAInmI1kYEeksnESJSQ== 2026-02-17T12:22:57.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQChXZRpAtwgLhAAr3akAInmI1kYEeksnESJSQ=="}' 2026-02-17T12:22:57.775 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1bc4b594-93b9-4ec8-b077-96fcd892174c -i td/test-erasure-eio/0/new.json 2026-02-17T12:22:57.877 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:22:57.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:22:57.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChXZRpAtwgLhAAr3akAInmI1kYEeksnESJSQ== --osd-uuid 1bc4b594-93b9-4ec8-b077-96fcd892174c 2026-02-17T12:22:57.901 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:57.900+0000 7f1c0f62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:57.903 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:57.901+0000 7f1c0f62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:57.904 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:57.902+0000 7f1c0f62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:22:57.904 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:57.903+0000 7f1c0f62f780 -1 bdev(0x556a20175c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:22:57.904 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:22:57.903+0000 7f1c0f62f780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:23:00.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:23:00.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:00.283 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:23:00.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:23:00.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:00.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:23:00.541 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:23:00.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:00.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:00.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:00.542 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:00.560 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:00.559+0000 7f4a2e32e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:00.562 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:00.560+0000 7f4a2e32e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:00.562 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:00.561+0000 7f4a2e32e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:00.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:00.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:00.738 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:00.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:00.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:00.924 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:01.925 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:01.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:01.926 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:01.926 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:01.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:01.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:02.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:02.147 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:02.145+0000 7f4a2e32e780 -1 Falling back to public interface 2026-02-17T12:23:03.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:03.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:03.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:03.118 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:03.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:03.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:03.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:04.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:04.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:04.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:04.305 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:04.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:04.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:04.501 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:05.086 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.085+0000 7f4a2e32e780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:23:05.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:05.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:05.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:05.503 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:05.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:05.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:05.696 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1032630132,v1:127.0.0.1:6803/1032630132] [v2:127.0.0.1:6804/1032630132,v1:127.0.0.1:6805/1032630132] exists,up 1bc4b594-93b9-4ec8-b077-96fcd892174c 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:05.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:05.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3950d3a5-9d00-44d5-8d58-e66fd147a553 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3950d3a5-9d00-44d5-8d58-e66fd147a553' 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 3950d3a5-9d00-44d5-8d58-e66fd147a553 2026-02-17T12:23:05.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:05.711 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCpXZRpotFbKhAAPpfGtXgmYu7Bq/BztBkkig== 2026-02-17T12:23:05.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCpXZRpotFbKhAAPpfGtXgmYu7Bq/BztBkkig=="}' 2026-02-17T12:23:05.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3950d3a5-9d00-44d5-8d58-e66fd147a553 -i td/test-erasure-eio/1/new.json 2026-02-17T12:23:05.904 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:05.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:23:05.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpXZRpotFbKhAAPpfGtXgmYu7Bq/BztBkkig== --osd-uuid 3950d3a5-9d00-44d5-8d58-e66fd147a553 2026-02-17T12:23:05.929 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.928+0000 7f50997ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:05.931 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.930+0000 7f50997ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:05.932 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.931+0000 7f50997ab780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:05.932 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.931+0000 7f50997ab780 -1 bdev(0x55e22b0fdc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:05.932 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:05.931+0000 7f50997ab780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:23:08.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:23:08.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:08.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:23:08.042 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:23:08.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:08.293 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:23:08.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:23:08.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:08.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:08.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:08.294 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:08.310 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:08.309+0000 7f6d03e21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:08.311 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:08.310+0000 7f6d03e21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:08.312 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:08.311+0000 7f6d03e21780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:08.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:23:08.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:08.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:23:08.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:08.483 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:08.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:08.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:08.484 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:08.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:08.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:08.671 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:09.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:09.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:09.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:09.673 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:09.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:09.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:09.860 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:09.913 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:09.912+0000 7f6d03e21780 -1 Falling back to public interface 2026-02-17T12:23:10.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:10.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:10.862 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:10.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:11.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:12.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:12.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:12.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:12.053 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:12.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:12.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:12.243 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:12.626 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:12.624+0000 7f6d03e21780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:23:13.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:13.244 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:13.244 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:13.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:13.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:13.245 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:13.431 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1537354239,v1:127.0.0.1:6811/1537354239] [v2:127.0.0.1:6812/1537354239,v1:127.0.0.1:6813/1537354239] exists,up 3950d3a5-9d00-44d5-8d58-e66fd147a553 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:23:13.432 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:13.433 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:13.434 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e868786b-958d-4903-9107-920c0b0816b1 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e868786b-958d-4903-9107-920c0b0816b1' 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 e868786b-958d-4903-9107-920c0b0816b1 2026-02-17T12:23:13.435 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:13.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCxXZRpxueSGhAAmKuqnNBnDCdtogxM0dlb2Q== 2026-02-17T12:23:13.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCxXZRpxueSGhAAmKuqnNBnDCdtogxM0dlb2Q=="}' 2026-02-17T12:23:13.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e868786b-958d-4903-9107-920c0b0816b1 -i td/test-erasure-eio/2/new.json 2026-02-17T12:23:13.638 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:13.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:23:13.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCxXZRpxueSGhAAmKuqnNBnDCdtogxM0dlb2Q== --osd-uuid e868786b-958d-4903-9107-920c0b0816b1 2026-02-17T12:23:13.664 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:13.663+0000 7fd6de41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:13.665 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:13.664+0000 7fd6de41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:13.666 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:13.665+0000 7fd6de41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:13.667 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:13.666+0000 7fd6de41b780 -1 bdev(0x5652d2ee3c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:13.667 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:13.666+0000 7fd6de41b780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:23:15.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:23:15.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:15.783 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:23:15.783 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:23:15.784 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:16.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:23:16.041 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:23:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:16.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:16.058 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:16.056+0000 7f177e427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:16.059 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:16.058+0000 7f177e427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:16.060 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:16.059+0000 7f177e427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:16.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:23:16.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:16.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:16.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:16.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:17.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:17.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:17.424 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:17.425 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:17.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:17.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:17.613 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:17.663 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:17.661+0000 7f177e427780 -1 Falling back to public interface 2026-02-17T12:23:18.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:18.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:18.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:18.615 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:18.615 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:18.615 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:18.802 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:19.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:19.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:19.804 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:19.805 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:19.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:19.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:19.997 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:20.604 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:20.602+0000 7f177e427780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:20.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:21.190 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3774519324,v1:127.0.0.1:6819/3774519324] [v2:127.0.0.1:6820/3774519324,v1:127.0.0.1:6821/3774519324] exists,up e868786b-958d-4903-9107-920c0b0816b1 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:21.191 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:21.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:23:21.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:21.194 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=644450ef-70e4-4d65-bc88-d9a2b0c7c5fa 2026-02-17T12:23:21.195 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 644450ef-70e4-4d65-bc88-d9a2b0c7c5fa' 2026-02-17T12:23:21.195 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 644450ef-70e4-4d65-bc88-d9a2b0c7c5fa 2026-02-17T12:23:21.195 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:21.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC5XZRpMhQ9DBAA9sZgw4YM9m22o7XcPmIy9Q== 2026-02-17T12:23:21.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC5XZRpMhQ9DBAA9sZgw4YM9m22o7XcPmIy9Q=="}' 2026-02-17T12:23:21.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 644450ef-70e4-4d65-bc88-d9a2b0c7c5fa -i td/test-erasure-eio/3/new.json 2026-02-17T12:23:21.401 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:21.409 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:23:21.410 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5XZRpMhQ9DBAA9sZgw4YM9m22o7XcPmIy9Q== --osd-uuid 644450ef-70e4-4d65-bc88-d9a2b0c7c5fa 2026-02-17T12:23:21.428 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:21.425+0000 7f4d0a19c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:21.429 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:21.427+0000 7f4d0a19c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:21.430 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:21.428+0000 7f4d0a19c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:21.430 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:21.428+0000 7f4d0a19c780 -1 bdev(0x563735a4dc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:21.430 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:21.428+0000 7f4d0a19c780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:23:23.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:23:23.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:23.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:23:23.546 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:23:23.546 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:23.800 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:23:23.801 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:23:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:23.801 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:23.817 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:23.814+0000 7f742b614780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:23.818 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:23.816+0000 7f742b614780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:23.819 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:23.817+0000 7f742b614780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:23.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:23.987 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:23.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:23.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:24.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:25.178 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:25.373 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:25.409 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:25.406+0000 7f742b614780 -1 Falling back to public interface 2026-02-17T12:23:26.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:26.374 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:26.375 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:26.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:26.375 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:26.376 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:26.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:27.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:27.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:27.569 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:27.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:27.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:27.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:27.760 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:28.356 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:28.355+0000 7f742b614780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:23:28.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:28.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:28.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:28.761 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:28.761 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:28.762 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:28.955 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:29.956 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:23:30.142 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1622742464,v1:127.0.0.1:6827/1622742464] [v2:127.0.0.1:6828/1622742464,v1:127.0.0.1:6829/1622742464] exists,up 644450ef-70e4-4d65-bc88-d9a2b0c7c5fa 2026-02-17T12:23:30.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:30.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:30.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:30.144 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:23:30.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:23:30.189 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:23:30.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:23:30.195 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:23:02.148+0000 7f4a2e32e780 0 load: jerasure load: lrc 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:270: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:271: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:23:30.196 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:23:30.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:23:30.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:23:30.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:23:30.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:23:30.737 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:23:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:23:31.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:23:31.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:23:31.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:23:31.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:23:31.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:23:31.749 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:23:31.749 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:23:31.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:23:31.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:23:31.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:23:31.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:23:31.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:23:31.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:23:31.837 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:23:31.838 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:23:31.838 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:23:32.033 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:23:32.034 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:23:32.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:23:32.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:23:32.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:23:32.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:23:32.095 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:23:32.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=47244640262 2026-02-17T12:23:32.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 47244640262 2026-02-17T12:23:32.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-47244640262' 2026-02-17T12:23:32.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:23:32.158 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:23:32.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=68719476741 2026-02-17T12:23:32.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 68719476741 2026-02-17T12:23:32.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-47244640262 2-68719476741' 2026-02-17T12:23:32.220 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:23:32.220 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215108 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215108 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-47244640262 2-68719476741 3-103079215108' 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:23:32.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:23:32.283 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:23:32.283 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:23:32.284 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:23:32.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:23:32.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:23:32.285 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:23:32.285 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:23:32.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-17T12:23:32.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:23:33.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:23:33.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:23:33.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-17T12:23:33.669 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:23:33.669 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-47244640262 2026-02-17T12:23:33.670 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:23:33.670 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:23:33.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-47244640262 2026-02-17T12:23:33.671 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:23:33.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=47244640262 2026-02-17T12:23:33.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 47244640262' 2026-02-17T12:23:33.672 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 47244640262 2026-02-17T12:23:33.672 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:23:33.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 47244640262 -lt 47244640262 2026-02-17T12:23:33.861 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:23:33.861 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-68719476741 2026-02-17T12:23:33.862 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:23:33.862 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:23:33.863 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-68719476741 2026-02-17T12:23:33.863 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:23:33.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=68719476741 2026-02-17T12:23:33.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 68719476741' 2026-02-17T12:23:33.864 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 68719476741 2026-02-17T12:23:33.865 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:23:34.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 68719476741 -lt 68719476741 2026-02-17T12:23:34.055 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:23:34.055 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-103079215108 2026-02-17T12:23:34.056 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:23:34.056 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:23:34.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-103079215108 2026-02-17T12:23:34.057 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:23:34.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215108 2026-02-17T12:23:34.058 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 103079215108' 2026-02-17T12:23:34.058 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 103079215108 2026-02-17T12:23:34.058 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:23:34.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215108 -lt 103079215108 2026-02-17T12:23:34.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:23:34.253 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:23:34.254 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:23:34.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:23:34.506 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:23:34.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:23:34.506 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:23:34.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:23:34.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:23:34.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:23:34.507 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:23:34.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:23:34.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:23:34.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:23:34.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:23:34.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:23:34.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:23:34.943 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:273: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:274: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-92877-1 2026-02-17T12:23:34.944 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-92877-1 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-92877-1 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:23:34.945 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-92877-1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-1 td/test-erasure-eio 1 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-1 2026-02-17T12:23:34.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-1 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-1 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:23:34.964 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-1 2026-02-17T12:23:35.154 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:23:35.154 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:23:35.155 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/1/type 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:23:35.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:23:35.158 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:23:35.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:23:35.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:23:35.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:23:35.207 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-92877-1 1 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-1 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-1 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:23:35.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-92877-1 td/test-erasure-eio/COPY 2026-02-17T12:23:35.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:23:35.273 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:23:35.274 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-17T12:23:35.275 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: shard_id=2 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-1 td/test-erasure-eio 2 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-1 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=2 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-1 2026-02-17T12:23:35.276 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:23:35.277 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-1 2026-02-17T12:23:35.277 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-1 2026-02-17T12:23:35.277 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=2 2026-02-17T12:23:35.466 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:23:35.467 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/2/type 2026-02-17T12:23:35.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=2 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.2 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:35.469 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:35.470 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.2.asok 2026-02-17T12:23:35.470 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:23:35.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:23:35.518 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:23:35.519 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:23:35.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.2 2026-02-17T12:23:35.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T12:23:35.519 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T12:23:35.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:35.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:35.520 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:35.521 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.2.asok 2026-02-17T12:23:35.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:23:35.521 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-92877-1 2 2026-02-17T12:23:35.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-1 fail 2026-02-17T12:23:35.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:23:35.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:23:35.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-1 2026-02-17T12:23:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:23:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:23:35.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-92877-1 td/test-erasure-eio/COPY 2026-02-17T12:23:35.585 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-eio-92877-1: (5) Input/output error 2026-02-17T12:23:35.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:23:35.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:184: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:23:35.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:23:35.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:23:35.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:23:35.859 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:23:35.866 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:23:36.069 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:23:36.077 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:23:36.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:23:36.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:23:36.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:23:36.186 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:23:36.187 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:23:36.188 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:23:36.188 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:23:36.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:23:36.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:23:36.190 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:23:36.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:23:36.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:23:36.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:23:36.191 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:23:36.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:23:36.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:23:36.193 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:23:36.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:23:36.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.202 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:23:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:23:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:23:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:23:36.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:23:36.205 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:23:36.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:23:36.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:23:36.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:23:36.207 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:23:36.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:23:36.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:23:36.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:23:36.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:23:36.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:23:36.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:23:36.209 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:23:36.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:23:36.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:23:36.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:23:36.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:23:36.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.212 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.212 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:23:36.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:23:36.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:23:36.213 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:23:36.214 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:23:36.214 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.215 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.215 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:23:36.216 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:23:36.217 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:23:36.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:23:36.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:23:36.218 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:23:36.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:23:36.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:36.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:36.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:36.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.237 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:36.237 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:23:36.262 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:23:36.265 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:23:36.266 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:23:36.267 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:23:36.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:23:36.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:23:36.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:23:36.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:23:36.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:23:36.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:23:36.355 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:23:36.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:23:36.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:36.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:36.457 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:36.457 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:36.458 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:36.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:36.458 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:23:36.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:23:36.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:23:36.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:23:36.571 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:23:36.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:23:37.580 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:23:37.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:23:37.621 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:23:37.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:23:37.628 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:23:36.255+0000 7fe10bf24d80 0 load: jerasure load: lrc 2026-02-17T12:23:37.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-17T12:23:37.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-17T12:23:37.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_missing: setup_osds 4 2026-02-17T12:23:37.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:23:37.629 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:23:37.630 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:23:37.630 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:23:37.634 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:37.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:37.636 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:37.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ae517759-30cd-4073-bb59-087d5e73f542 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ae517759-30cd-4073-bb59-087d5e73f542' 2026-02-17T12:23:37.638 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 ae517759-30cd-4073-bb59-087d5e73f542 2026-02-17T12:23:37.639 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:37.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDJXZRpKtBvJhAAkBOUuPmEXlo7hUAVISyD9g== 2026-02-17T12:23:37.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDJXZRpKtBvJhAAkBOUuPmEXlo7hUAVISyD9g=="}' 2026-02-17T12:23:37.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ae517759-30cd-4073-bb59-087d5e73f542 -i td/test-erasure-eio/0/new.json 2026-02-17T12:23:37.750 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:37.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:23:37.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJXZRpKtBvJhAAkBOUuPmEXlo7hUAVISyD9g== --osd-uuid ae517759-30cd-4073-bb59-087d5e73f542 2026-02-17T12:23:37.775 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:37.774+0000 7f0742cfe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:37.776 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:37.775+0000 7f0742cfe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:37.777 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:37.776+0000 7f0742cfe780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:37.778 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:37.776+0000 7f0742cfe780 -1 bdev(0x55a4a62b7c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:37.778 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:37.776+0000 7f0742cfe780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:23:40.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:23:40.157 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:40.158 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:23:40.158 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:23:40.159 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:40.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:40.435 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:40.433+0000 7f4a4282a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:40.436 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:40.435+0000 7f4a4282a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:40.437 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:40.436+0000 7f4a4282a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:40.607 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:40.608 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:40.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:40.609 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:40.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:41.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:41.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:41.798 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:41.799 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:41.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:41.799 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:41.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:42.021 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:42.020+0000 7f4a4282a780 -1 Falling back to public interface 2026-02-17T12:23:42.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:42.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:42.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:42.987 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:42.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:42.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:43.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:44.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:44.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:44.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:44.174 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:44.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:44.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:44.362 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:44.955 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:44.953+0000 7f4a4282a780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:23:45.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:45.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:45.363 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:45.363 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:45.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:45.364 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:45.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:46.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:46.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:46.555 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:23:46.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:23:46.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:46.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:23:46.743 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3588406685,v1:127.0.0.1:6803/3588406685] [v2:127.0.0.1:6804/3588406685,v1:127.0.0.1:6805/3588406685] exists,up ae517759-30cd-4073-bb59-087d5e73f542 2026-02-17T12:23:46.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:46.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:46.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:46.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:46.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:46.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:46.750 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4f5c052a-794a-4ca3-b90a-db48faead000 2026-02-17T12:23:46.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 4f5c052a-794a-4ca3-b90a-db48faead000' 2026-02-17T12:23:46.751 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 4f5c052a-794a-4ca3-b90a-db48faead000 2026-02-17T12:23:46.751 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:46.758 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDSXZRpeCImLRAAX8mBgnayPM6ZPU/7DK/Plw== 2026-02-17T12:23:46.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDSXZRpeCImLRAAX8mBgnayPM6ZPU/7DK/Plw=="}' 2026-02-17T12:23:46.759 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4f5c052a-794a-4ca3-b90a-db48faead000 -i td/test-erasure-eio/1/new.json 2026-02-17T12:23:46.952 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:46.959 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:23:46.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSXZRpeCImLRAAX8mBgnayPM6ZPU/7DK/Plw== --osd-uuid 4f5c052a-794a-4ca3-b90a-db48faead000 2026-02-17T12:23:46.978 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:46.976+0000 7f281ea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:46.979 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:46.978+0000 7f281ea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:46.980 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:46.979+0000 7f281ea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:46.981 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:46.979+0000 7f281ea0d780 -1 bdev(0x5608c455bc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:46.981 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:46.979+0000 7f281ea0d780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:23:49.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:23:49.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:49.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:23:49.105 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:23:49.105 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:49.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:23:49.360 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:23:49.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:49.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:49.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:49.361 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:49.376 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:49.375+0000 7f7946c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:49.378 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:49.376+0000 7f7946c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:49.378 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:49.377+0000 7f7946c0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:49.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:49.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:49.736 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:50.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:50.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:50.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:50.738 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:50.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:50.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:50.927 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:50.969 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:50.967+0000 7f7946c0c780 -1 Falling back to public interface 2026-02-17T12:23:51.928 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:51.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:51.929 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:51.929 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:23:51.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:51.930 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:52.117 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:53.118 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:53.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:53.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:23:53.119 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:23:53.119 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:53.120 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:53.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:53.932 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:53.931+0000 7f7946c0c780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:23:54.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:54.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:54.312 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:23:54.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:23:54.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:54.313 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:54.503 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:55.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:55.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:55.504 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:23:55.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:23:55.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:55.505 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:23:55.702 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3570886674,v1:127.0.0.1:6811/3570886674] [v2:127.0.0.1:6812/3570886674,v1:127.0.0.1:6813/3570886674] exists,up 4f5c052a-794a-4ca3-b90a-db48faead000 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:23:55.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:23:55.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:23:55.705 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:23:55.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2eb08135-a12c-47a8-bbe3-cb7e0e123bcb 2026-02-17T12:23:55.706 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb' 2026-02-17T12:23:55.706 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb 2026-02-17T12:23:55.707 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:23:55.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDbXZRpHmbGKhAANTAWycETcfVoTJgYIteXOA== 2026-02-17T12:23:55.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDbXZRpHmbGKhAANTAWycETcfVoTJgYIteXOA=="}' 2026-02-17T12:23:55.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb -i td/test-erasure-eio/2/new.json 2026-02-17T12:23:55.911 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:23:55.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:23:55.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDbXZRpHmbGKhAANTAWycETcfVoTJgYIteXOA== --osd-uuid 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb 2026-02-17T12:23:55.937 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:55.936+0000 7f7f2b79c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:55.938 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:55.937+0000 7f7f2b79c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:55.939 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:55.938+0000 7f7f2b79c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:55.940 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:55.938+0000 7f7f2b79c780 -1 bdev(0x55c24f5f7c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:23:55.940 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:55.938+0000 7f7f2b79c780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:23:58.050 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:23:58.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:23:58.051 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:23:58.052 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:23:58.052 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:23:58.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:23:58.322 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:23:58.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:23:58.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:23:58.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:23:58.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:23:58.339 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:58.338+0000 7ff5e8ff9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:58.340 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:58.339+0000 7ff5e8ff9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:58.341 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:58.340+0000 7ff5e8ff9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:23:58.510 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:23:58.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:58.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:58.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:59.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:23:59.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:23:59.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:23:59.703 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:23:59.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:23:59.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:23:59.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:23:59.921 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:23:59.919+0000 7ff5e8ff9780 -1 Falling back to public interface 2026-02-17T12:24:00.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:00.890 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:00.891 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:24:00.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:00.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:00.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:01.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:02.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:02.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:02.876 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:02.874+0000 7ff5e8ff9780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:24:03.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:03.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:03.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:24:03.271 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:24:03.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:03.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:03.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:04.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:04.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:04.461 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:24:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:24:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:04.652 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2606593651,v1:127.0.0.1:6819/2606593651] [v2:127.0.0.1:6820/2606593651,v1:127.0.0.1:6821/2606593651] exists,up 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb 2026-02-17T12:24:04.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:04.653 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:24:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:24:04.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:24:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=553e0859-badc-4a15-a150-5f80d7f1008a 2026-02-17T12:24:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 553e0859-badc-4a15-a150-5f80d7f1008a' 2026-02-17T12:24:04.655 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 553e0859-badc-4a15-a150-5f80d7f1008a 2026-02-17T12:24:04.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:24:04.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDkXZRpzn68JxAAjYQs8Q2Lyd6Yp0HwVTUiXQ== 2026-02-17T12:24:04.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDkXZRpzn68JxAAjYQs8Q2Lyd6Yp0HwVTUiXQ=="}' 2026-02-17T12:24:04.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 553e0859-badc-4a15-a150-5f80d7f1008a -i td/test-erasure-eio/3/new.json 2026-02-17T12:24:04.863 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:24:04.871 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:24:04.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkXZRpzn68JxAAjYQs8Q2Lyd6Yp0HwVTUiXQ== --osd-uuid 553e0859-badc-4a15-a150-5f80d7f1008a 2026-02-17T12:24:04.889 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:04.887+0000 7f1bffc13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:04.890 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:04.888+0000 7f1bffc13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:04.891 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:04.889+0000 7f1bffc13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:04.891 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:04.889+0000 7f1bffc13780 -1 bdev(0x56148fc4dc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:24:04.891 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:04.889+0000 7f1bffc13780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:24:07.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:24:07.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:24:07.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:24:07.010 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:24:07.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:24:07.270 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:24:07.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:24:07.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:24:07.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:24:07.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:24:07.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:24:07.287 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:07.285+0000 7f90f73f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:07.289 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:07.286+0000 7f90f73f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:07.289 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:07.287+0000 7f90f73f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:07.470 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:07.471 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:24:07.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:24:07.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:07.472 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:07.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:08.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:08.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:08.663 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:24:08.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:24:08.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:08.664 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:08.852 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:08.877 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:08.875+0000 7f90f73f9780 -1 Falling back to public interface 2026-02-17T12:24:09.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:09.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:09.855 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:24:09.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:09.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:09.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:10.041 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:11.042 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:11.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:11.043 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:24:11.043 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:24:11.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:11.044 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:11.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:12.075 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:12.074+0000 7f90f73f9780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:24:12.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:12.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:12.230 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:24:12.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:24:12.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:12.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:12.423 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:13.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:13.425 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:13.425 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:24:13.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:24:13.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:13.426 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:24:13.617 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2539123992,v1:127.0.0.1:6827/2539123992] [v2:127.0.0.1:6828/2539123992,v1:127.0.0.1:6829/2539123992] exists,up 553e0859-badc-4a15-a150-5f80d7f1008a 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:24:13.618 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:24:13.619 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:13.619 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:13.619 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:24:13.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:24:13.619 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:24:13.664 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:24:13.668 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:24:13.671 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:23:42.023+0000 7f4a4282a780 0 load: jerasure load: lrc 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:285: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:286: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:24:13.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:24:13.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:24:13.673 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:24:13.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:24:13.919 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:24:14.212 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:24:14.219 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:24:15.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:24:15.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:24:15.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:24:15.221 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:24:15.222 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:24:15.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:24:15.223 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:24:15.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:24:15.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:24:15.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:24:15.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:24:15.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:24:15.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:24:15.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:24:15.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:24:15.312 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:15.495 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:24:15.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:24:15.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:24:15.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:24:15.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:15.561 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:24:15.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607559 2026-02-17T12:24:15.623 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607559 2026-02-17T12:24:15.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-17T12:24:15.624 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:15.624 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:24:15.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444037 2026-02-17T12:24:15.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444037 2026-02-17T12:24:15.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-73014444037' 2026-02-17T12:24:15.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:15.686 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182403 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182403 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-73014444037 3-107374182403' 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:24:15.746 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:15.747 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:24:15.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:24:15.748 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:15.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:24:15.749 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:24:15.749 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:24:15.750 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:15.939 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:24:15.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:24:16.940 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:24:16.941 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:17.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803785 -lt 25769803784 2026-02-17T12:24:17.130 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:17.130 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607559 2026-02-17T12:24:17.130 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:17.131 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:24:17.132 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607559 2026-02-17T12:24:17.132 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:17.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607559 2026-02-17T12:24:17.133 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-17T12:24:17.133 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607559 2026-02-17T12:24:17.134 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:24:17.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-17T12:24:17.324 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:17.324 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444037 2026-02-17T12:24:17.325 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:17.325 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:24:17.326 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444037 2026-02-17T12:24:17.326 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:17.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444037 2026-02-17T12:24:17.327 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444037' 2026-02-17T12:24:17.327 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 73014444037 2026-02-17T12:24:17.328 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:24:17.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444037 -lt 73014444037 2026-02-17T12:24:17.513 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:17.513 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182403 2026-02-17T12:24:17.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:17.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:24:17.514 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182403 2026-02-17T12:24:17.515 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:17.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182403 2026-02-17T12:24:17.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-17T12:24:17.516 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 107374182403 2026-02-17T12:24:17.516 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:24:17.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-17T12:24:17.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:24:17.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:17.700 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:17.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:24:17.952 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:24:17.953 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:24:18.138 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:24:18.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:24:18.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:18.138 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:18.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:24:18.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:24:18.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_missing: local shard_id=1 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:289: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-92877-1 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-92877-1 2026-02-17T12:24:18.394 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-92877-1 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:24:18.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-92877-1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-92877-1 td/test-erasure-eio 1 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-17T12:24:18.414 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-92877-1 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-92877-1 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-remove-92877-1 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-remove-92877-1 2026-02-17T12:24:18.415 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:24:18.610 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-92877-1 remove 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:24:18.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-92877-1 remove 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:24:18.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-92877-1 remove 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-17T12:24:19.918 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-92877-1 remove 2026-02-17T12:24:20.492 INFO:tasks.workunit.client.0.trial044.stdout:remove 1#2:5d8332ff:::obj-remove-92877-1:head# 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:24:21.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:21.009 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:24:21.010 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:24:21.011 INFO:tasks.workunit.client.0.trial044.stderr:start osd.1 2026-02-17T12:24:21.011 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:24:21.011 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-17T12:24:21.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:24:21.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:24:21.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:24:21.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:24:21.028 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:21.027+0000 7fabaf23f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:21.029 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:21.028+0000 7fabaf23f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:21.030 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:21.029+0000 7fabaf23f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:21.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:24:21.202 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:24:21.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:21.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:21.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:22.367 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:22.366+0000 7fabaf23f780 -1 Falling back to public interface 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:22.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:22.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:23.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:23.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:23.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:23.583 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:24:23.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:23.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:23.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:24.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:24.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:24.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:24:24.778 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:24:24.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:24.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:24.969 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:25.050 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:25.049+0000 7fabaf23f780 -1 osd.1 33 log_to_monitors true 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:25.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:26.161 INFO:tasks.workunit.client.0.trial044.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [12,33) [v2:127.0.0.1:6810/1181453055,v1:127.0.0.1:6811/1181453055] [v2:127.0.0.1:6812/1181453055,v1:127.0.0.1:6813/1181453055] exists,up 4f5c052a-794a-4ca3-b90a-db48faead000 2026-02-17T12:24:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:24:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:24:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:24:26.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:24:26.162 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:24:26.250 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:24:26.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:24:26.440 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:26.440 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:24:26.440 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:24:26.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:24:26.441 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:26.441 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:24:26.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:24:26.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:24:26.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:24:26.504 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:26.504 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:24:26.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789955 2026-02-17T12:24:26.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789955 2026-02-17T12:24:26.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-158913789955' 2026-02-17T12:24:26.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:26.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:24:26.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444041 2026-02-17T12:24:26.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444041 2026-02-17T12:24:26.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-158913789955 2-73014444041' 2026-02-17T12:24:26.631 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:26.632 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:24:26.690 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182407 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182407 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-158913789955 2-73014444041 3-107374182407' 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:26.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:24:26.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:24:26.692 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:26.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:24:26.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:24:26.693 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803789 2026-02-17T12:24:26.694 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:26.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:24:26.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:24:27.884 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:24:27.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:28.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:24:28.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:24:29.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:24:29.078 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:29.267 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:24:29.267 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:29.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789955 2026-02-17T12:24:29.268 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:29.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:24:29.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789955 2026-02-17T12:24:29.269 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:29.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789955 2026-02-17T12:24:29.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789955' 2026-02-17T12:24:29.271 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 158913789955 2026-02-17T12:24:29.271 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:24:29.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-17T12:24:29.457 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:29.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444041 2026-02-17T12:24:29.457 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:29.458 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:24:29.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444041 2026-02-17T12:24:29.459 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:29.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444041 2026-02-17T12:24:29.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444041' 2026-02-17T12:24:29.460 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 73014444041 2026-02-17T12:24:29.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:24:29.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444041 -lt 73014444041 2026-02-17T12:24:29.650 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:29.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182407 2026-02-17T12:24:29.651 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:29.652 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:24:29.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182407 2026-02-17T12:24:29.652 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:29.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182407 2026-02-17T12:24:29.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182407' 2026-02-17T12:24:29.654 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 107374182407 2026-02-17T12:24:29.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:24:29.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182408 -lt 107374182407 2026-02-17T12:24:29.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:24:29.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:29.844 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:30.095 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:24:30.096 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:24:30.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:24:30.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:24:30.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:30.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-92877-1 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:24:30.530 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:24:30.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-92877-1 2026-02-17T12:24:30.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:24:30.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:24:30.531 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-92877-1 td/test-erasure-eio/COPY 2026-02-17T12:24:30.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:24:30.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:24:30.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-17T12:24:30.552 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: shard_id=2 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-92877-1 td/test-erasure-eio 2 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-92877-1 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-92877-1 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:24:30.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-remove-92877-1 2026-02-17T12:24:30.554 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-remove-92877-1 2026-02-17T12:24:30.554 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:24:30.744 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:24:30.744 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-92877-1 remove 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/test-erasure-eio 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-92877-1 remove 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:24:30.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:24:30.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:24:32.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:24:32.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-92877-1 remove 2026-02-17T12:24:32.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-17T12:24:32.053 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:24:32.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:24:32.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:24:32.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-17T12:24:32.054 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-92877-1 remove 2026-02-17T12:24:32.623 INFO:tasks.workunit.client.0.trial044.stdout:remove 2#2:5d8332ff:::obj-remove-92877-1:head# 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:24:33.142 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:33.143 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:24:33.144 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:24:33.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:24:33.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:24:33.145 INFO:tasks.workunit.client.0.trial044.stderr:start osd.2 2026-02-17T12:24:33.145 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:24:33.145 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:24:33.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:24:33.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:24:33.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:24:33.146 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:24:33.158 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:33.157+0000 7f569ec16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:33.159 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:33.158+0000 7f569ec16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:33.160 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:33.159+0000 7f569ec16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:33.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:0 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:33.336 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:33.525 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:34.242 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:34.241+0000 7f569ec16780 -1 Falling back to public interface 2026-02-17T12:24:34.526 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:34.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:34.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:24:34.527 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:34.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:34.527 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:34.714 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:35.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:35.716 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:35.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:35.717 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:24:35.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:35.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:35.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:3 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:36.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:36.946 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:36.945+0000 7f569ec16780 -1 osd.2 39 log_to_monitors true 2026-02-17T12:24:37.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:4 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:38.111 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:24:38.308 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 up in weight 1 up_from 43 up_thru 34 down_at 40 last_clean_interval [17,39) [v2:127.0.0.1:6818/630471604,v1:127.0.0.1:6819/630471604] [v2:127.0.0.1:6820/630471604,v1:127.0.0.1:6821/630471604] exists,up 2eb08135-a12c-47a8-bbe3-cb7e0e123bcb 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:24:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:24:38.310 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:24:38.393 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:24:38.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:38.581 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:24:38.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803793 2026-02-17T12:24:38.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803793 2026-02-17T12:24:38.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793' 2026-02-17T12:24:38.643 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:38.643 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:24:38.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789959 2026-02-17T12:24:38.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789959 2026-02-17T12:24:38.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-158913789959' 2026-02-17T12:24:38.704 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:38.704 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:24:38.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593731 2026-02-17T12:24:38.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593731 2026-02-17T12:24:38.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-158913789959 2-184683593731' 2026-02-17T12:24:38.765 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:24:38.766 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182412 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182412 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803793 1-158913789959 2-184683593731 3-107374182412' 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803793 2026-02-17T12:24:38.828 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:38.829 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:24:38.829 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803793 2026-02-17T12:24:38.829 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:38.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803793 2026-02-17T12:24:38.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-17T12:24:38.831 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.0 seq 25769803793 2026-02-17T12:24:38.831 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:39.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-17T12:24:39.020 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:24:40.021 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:24:40.022 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:40.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-17T12:24:40.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:24:41.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:24:41.209 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:24:41.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-17T12:24:41.400 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:41.401 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789959 2026-02-17T12:24:41.401 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:41.401 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:24:41.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789959 2026-02-17T12:24:41.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:41.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789959 2026-02-17T12:24:41.403 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789959' 2026-02-17T12:24:41.403 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.1 seq 158913789959 2026-02-17T12:24:41.403 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:24:41.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789960 -lt 158913789959 2026-02-17T12:24:41.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:41.593 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-184683593731 2026-02-17T12:24:41.594 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:41.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:24:41.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-184683593731 2026-02-17T12:24:41.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:41.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593731 2026-02-17T12:24:41.597 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-17T12:24:41.597 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.2 seq 184683593731 2026-02-17T12:24:41.597 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:24:41.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-17T12:24:41.786 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:24:41.787 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-107374182412 2026-02-17T12:24:41.787 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:24:41.787 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:24:41.788 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-107374182412 2026-02-17T12:24:41.788 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:24:41.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182412 2026-02-17T12:24:41.789 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-17T12:24:41.789 INFO:tasks.workunit.client.0.trial044.stderr:waiting osd.3 seq 107374182412 2026-02-17T12:24:41.789 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:24:41.979 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182412 -lt 107374182412 2026-02-17T12:24:41.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:24:41.979 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:41.980 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:42.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:24:42.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:24:42.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:24:42.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:24:42.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:24:42.235 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:24:42.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:24:42.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:24:42.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:24:42.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:24:42.422 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:24:42.422 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:24:42.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:24:42.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:24:42.678 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-92877-1 fail 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-92877-1 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-17T12:24:42.679 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-remove-92877-1 td/test-erasure-eio/COPY 2026-02-17T12:24:42.695 INFO:tasks.workunit.client.0.trial044.stderr:error getting pool-jerasure/obj-remove-92877-1: (5) Input/output error 2026-02-17T12:24:42.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-17T12:24:42.696 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:184: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-17T12:24:42.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:24:42.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:24:42.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:24:42.883 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:24:42.891 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:24:43.088 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:24:43.096 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:24:43.096 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:24:43.097 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:24:43.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:24:43.205 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:24:43.206 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:24:43.206 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:24:43.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:24:43.208 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:24:43.208 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:24:43.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:24:43.209 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:24:43.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:24:43.210 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:24:43.210 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:24:43.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:24:43.211 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:24:43.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:24:43.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.223 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.223 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:24:43.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:24:43.227 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:24:43.227 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:24:43.228 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:24:43.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:24:43.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:24:43.229 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:24:43.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:24:43.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:24:43.230 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:24:43.231 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:24:43.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:24:43.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:24:43.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:24:43.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:24:43.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.234 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:24:43.234 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:24:43.235 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:24:43.235 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:24:43.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:24:43.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.236 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.236 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:24:43.237 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:24:43.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:24:43.238 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:24:43.239 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:24:43.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.259 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:43.260 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:24:43.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:24:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:24:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:24:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:24:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:24:43.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:24:43.287 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:24:43.332 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:24:43.332 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:24:43.332 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:24:43.332 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:24:43.332 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.333 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:24:43.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:24:43.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:24:43.377 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:43.479 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:43.480 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:43.480 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:24:43.480 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:24:43.497 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:24:43.498 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:24:43.592 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:24:43.598 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:24:44.599 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:24:44.600 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:24:44.641 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:24:44.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:24:44.647 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:24:43.275+0000 7f23237dad80 0 load: jerasure load: lrc 2026-02-17T12:24:44.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-17T12:24:44.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-17T12:24:44.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-17T12:24:44.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:333: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-17T12:24:44.648 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:24:44.649 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:24:44.649 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:24:44.649 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:24:44.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:24:44.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:24:44.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:24:44.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:24:44.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:24:44.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:24:44.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:24:44.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=028f26a0-dbc5-40d1-840e-54784246b184 2026-02-17T12:24:44.657 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 028f26a0-dbc5-40d1-840e-54784246b184 2026-02-17T12:24:44.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 028f26a0-dbc5-40d1-840e-54784246b184' 2026-02-17T12:24:44.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:24:44.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAMXpRpYGKgJxAAC2Of1ZDcAVvhsY7ruDMMfw== 2026-02-17T12:24:44.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAMXpRpYGKgJxAAC2Of1ZDcAVvhsY7ruDMMfw=="}' 2026-02-17T12:24:44.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 028f26a0-dbc5-40d1-840e-54784246b184 -i td/test-erasure-eio/0/new.json 2026-02-17T12:24:44.767 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:24:44.773 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:24:44.774 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMXpRpYGKgJxAAC2Of1ZDcAVvhsY7ruDMMfw== --osd-uuid 028f26a0-dbc5-40d1-840e-54784246b184 2026-02-17T12:24:44.791 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:44.789+0000 7f99d5ce5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:44.792 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:44.790+0000 7f99d5ce5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:44.793 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:44.791+0000 7f99d5ce5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:44.794 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:44.791+0000 7f99d5ce5780 -1 bdev(0x559b49cb3c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:24:44.794 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:44.791+0000 7f99d5ce5780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:24:47.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:24:47.167 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:24:47.168 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:24:47.168 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:24:47.169 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:24:47.435 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:24:47.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:24:47.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:24:47.436 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:24:47.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:24:47.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:24:47.452 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:47.450+0000 7fd499912780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:47.454 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:47.451+0000 7fd499912780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:47.454 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:47.452+0000 7fd499912780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:47.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:24:47.627 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:24:47.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:47.627 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:47.812 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:24:48.814 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:24:48.814 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:48.815 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:49.001 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:49.035 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:49.033+0000 7fd499912780 -1 Falling back to public interface 2026-02-17T12:24:50.002 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:50.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:50.003 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:50.003 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:24:50.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:50.004 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:50.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:51.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:51.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:51.993 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:51.991+0000 7fd499912780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:52.390 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:52.580 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:53.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:53.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:53.582 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:24:53.583 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:24:53.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:53.583 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:24:53.772 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1516587930,v1:127.0.0.1:6803/1516587930] [v2:127.0.0.1:6804/1516587930,v1:127.0.0.1:6805/1516587930] exists,up 028f26a0-dbc5-40d1-840e-54784246b184 2026-02-17T12:24:53.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:24:53.776 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:24:53.777 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:24:53.778 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:24:53.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=04c87e07-e91c-4abd-9a6d-b7e8c35bfc33 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 04c87e07-e91c-4abd-9a6d-b7e8c35bfc33' 2026-02-17T12:24:53.780 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 04c87e07-e91c-4abd-9a6d-b7e8c35bfc33 2026-02-17T12:24:53.781 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:24:53.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAVXpRphY/pLhAAaauti8ZgKxVWbEfJhRELyA== 2026-02-17T12:24:53.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAVXpRphY/pLhAAaauti8ZgKxVWbEfJhRELyA=="}' 2026-02-17T12:24:53.788 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 04c87e07-e91c-4abd-9a6d-b7e8c35bfc33 -i td/test-erasure-eio/1/new.json 2026-02-17T12:24:53.979 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:24:53.985 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:24:53.986 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVXpRphY/pLhAAaauti8ZgKxVWbEfJhRELyA== --osd-uuid 04c87e07-e91c-4abd-9a6d-b7e8c35bfc33 2026-02-17T12:24:54.003 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:54.002+0000 7fe62860c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:54.005 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:54.003+0000 7fe62860c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:54.006 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:54.004+0000 7fe62860c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:54.006 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:54.005+0000 7fe62860c780 -1 bdev(0x5629cbb57c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:24:54.006 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:54.005+0000 7fe62860c780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:24:56.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:24:56.134 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:24:56.135 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:24:56.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:24:56.135 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:24:56.388 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:24:56.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:24:56.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:24:56.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:24:56.388 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:24:56.389 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:24:56.405 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:56.403+0000 7ff524027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:56.406 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:56.404+0000 7ff524027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:56.406 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:56.405+0000 7ff524027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:56.578 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:56.767 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:57.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:57.768 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:57.769 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:24:57.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:24:57.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:57.770 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:57.957 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:24:57.990 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:24:57.988+0000 7ff524027780 -1 Falling back to public interface 2026-02-17T12:24:58.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:24:58.958 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:24:58.959 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:24:58.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:24:58.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:24:58.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:24:59.152 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:00.153 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:00.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:00.932 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:00.931+0000 7ff524027780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:25:01.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:01.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:01.351 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:01.351 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:01.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:01.352 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:01.546 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2585110287,v1:127.0.0.1:6811/2585110287] [v2:127.0.0.1:6812/2585110287,v1:127.0.0.1:6813/2585110287] exists,up 04c87e07-e91c-4abd-9a6d-b7e8c35bfc33 2026-02-17T12:25:01.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:01.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:01.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:01.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=faf4900f-5452-4707-8490-ef9b0edb5437 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 faf4900f-5452-4707-8490-ef9b0edb5437' 2026-02-17T12:25:01.550 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 faf4900f-5452-4707-8490-ef9b0edb5437 2026-02-17T12:25:01.551 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:25:01.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAdXpRpEWt+IRAApiFR8xdodOkbQwETk5RG4g== 2026-02-17T12:25:01.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAdXpRpEWt+IRAApiFR8xdodOkbQwETk5RG4g=="}' 2026-02-17T12:25:01.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new faf4900f-5452-4707-8490-ef9b0edb5437 -i td/test-erasure-eio/2/new.json 2026-02-17T12:25:01.756 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:01.763 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:25:01.764 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAdXpRpEWt+IRAApiFR8xdodOkbQwETk5RG4g== --osd-uuid faf4900f-5452-4707-8490-ef9b0edb5437 2026-02-17T12:25:01.781 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:01.780+0000 7fa490c0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:01.782 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:01.781+0000 7fa490c0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:01.783 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:01.782+0000 7fa490c0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:01.784 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:01.783+0000 7fa490c0e780 -1 bdev(0x55acb08dfc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:25:01.784 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:01.783+0000 7fa490c0e780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:25:04.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:25:04.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:25:04.154 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:25:04.155 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:25:04.155 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:25:04.405 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:25:04.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:25:04.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:04.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:25:04.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:25:04.406 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:25:04.422 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:04.421+0000 7f84c4753780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:04.423 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:04.422+0000 7f84c4753780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:04.424 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:04.423+0000 7f84c4753780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:04.595 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:04.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:04.596 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:04.794 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:05.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:05.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:05.795 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:05.796 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:05.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:05.796 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:05.987 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:06.028 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:06.026+0000 7f84c4753780 -1 Falling back to public interface 2026-02-17T12:25:06.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:06.988 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:06.989 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:25:06.989 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:06.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:06.990 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:07.174 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:08.175 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:08.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:08.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:08.176 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:08.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:08.176 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:08.365 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:08.982 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:08.981+0000 7f84c4753780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:09.366 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:09.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:10.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:10.562 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:10.562 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:25:10.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:25:10.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:10.563 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:10.753 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4099426674,v1:127.0.0.1:6819/4099426674] [v2:127.0.0.1:6820/4099426674,v1:127.0.0.1:6821/4099426674] exists,up faf4900f-5452-4707-8490-ef9b0edb5437 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:10.754 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:10.755 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:25:10.756 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:25:10.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=672bc5af-f2b5-4967-9168-faa4f11008cf 2026-02-17T12:25:10.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 672bc5af-f2b5-4967-9168-faa4f11008cf' 2026-02-17T12:25:10.757 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 672bc5af-f2b5-4967-9168-faa4f11008cf 2026-02-17T12:25:10.757 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:25:10.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAmXpRpifXLLRAAS8PBgzzC1ktEFqsNQm7m1g== 2026-02-17T12:25:10.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAmXpRpifXLLRAAS8PBgzzC1ktEFqsNQm7m1g=="}' 2026-02-17T12:25:10.769 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 672bc5af-f2b5-4967-9168-faa4f11008cf -i td/test-erasure-eio/3/new.json 2026-02-17T12:25:10.963 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:10.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:25:10.971 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmXpRpifXLLRAAS8PBgzzC1ktEFqsNQm7m1g== --osd-uuid 672bc5af-f2b5-4967-9168-faa4f11008cf 2026-02-17T12:25:10.988 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:10.987+0000 7f1633a82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:10.990 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:10.989+0000 7f1633a82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:10.991 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:10.990+0000 7f1633a82780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:10.991 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:10.990+0000 7f1633a82780 -1 bdev(0x556338bfdc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:25:10.991 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:10.990+0000 7f1633a82780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:25:13.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:25:13.108 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:25:13.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:25:13.109 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:25:13.109 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:25:13.369 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:25:13.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:25:13.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:13.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:25:13.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:25:13.370 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:25:13.385 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:13.384+0000 7f98e1e14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:13.387 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:13.385+0000 7f98e1e14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:13.388 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:13.386+0000 7f98e1e14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:13.559 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:13.560 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:13.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:13.561 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:13.744 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:14.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:14.745 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:14.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:14.746 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:14.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:14.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:14.934 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:14.971 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:14.969+0000 7f98e1e14780 -1 Falling back to public interface 2026-02-17T12:25:15.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:15.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:15.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:25:15.936 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:15.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:15.937 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:16.121 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:17.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:17.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:17.122 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:17.122 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:17.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:17.123 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:17.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:17.929 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:17.927+0000 7f98e1e14780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:25:18.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:18.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:18.311 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:18.312 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:18.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:18.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:18.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:19.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:19.510 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:19.511 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:25:19.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:25:19.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:19.511 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:25:19.697 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/653882033,v1:127.0.0.1:6827/653882033] [v2:127.0.0.1:6828/653882033,v1:127.0.0.1:6829/653882033] exists,up 672bc5af-f2b5-4967-9168-faa4f11008cf 2026-02-17T12:25:19.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:19.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:19.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:19.698 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:25:19.698 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:25:19.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:25:19.744 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:25:19.748 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:25:19.750 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:24:49.036+0000 7fd499912780 0 load: jerasure load: lrc 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:335: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:25:19.751 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:25:19.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:25:19.998 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:25:20.298 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:25:20.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:25:21.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:25:21.307 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:25:21.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:25:21.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:25:21.308 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:25:21.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:25:21.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:25:21.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:25:21.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:25:21.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:25:21.399 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:25:21.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:25:21.592 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:25:21.593 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:25:21.593 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:25:21.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:25:21.593 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:21.593 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:25:21.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:25:21.657 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:25:21.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:25:21.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:21.658 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:25:21.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607558 2026-02-17T12:25:21.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607558 2026-02-17T12:25:21.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558' 2026-02-17T12:25:21.719 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:21.719 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:25:21.779 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444038 2026-02-17T12:25:21.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444038 2026-02-17T12:25:21.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038' 2026-02-17T12:25:21.780 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:21.780 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:25:21.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215107 2026-02-17T12:25:21.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215107 2026-02-17T12:25:21.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038 3-103079215107' 2026-02-17T12:25:21.841 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:21.842 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:25:21.842 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:21.843 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:25:21.843 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:25:21.843 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:21.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:25:21.844 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:25:21.844 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:25:21.845 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:22.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:25:22.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:25:23.039 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:25:23.040 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:23.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:25:23.228 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:25:24.229 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:25:24.230 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:24.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803785 -lt 25769803784 2026-02-17T12:25:24.417 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:24.418 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607558 2026-02-17T12:25:24.418 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:24.419 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:25:24.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607558 2026-02-17T12:25:24.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:24.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607558 2026-02-17T12:25:24.421 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607558 2026-02-17T12:25:24.421 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-17T12:25:24.421 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:25:24.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607559 -lt 51539607558 2026-02-17T12:25:24.611 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:24.611 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-73014444038 2026-02-17T12:25:24.612 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:24.612 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:25:24.613 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-73014444038 2026-02-17T12:25:24.613 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:24.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444038 2026-02-17T12:25:24.614 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 73014444038 2026-02-17T12:25:24.614 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-17T12:25:24.615 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:25:24.805 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-17T12:25:24.806 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:24.806 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-103079215107 2026-02-17T12:25:24.806 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:24.807 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:25:24.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-103079215107 2026-02-17T12:25:24.808 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:24.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215107 2026-02-17T12:25:24.809 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 103079215107 2026-02-17T12:25:24.809 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 103079215107' 2026-02-17T12:25:24.809 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:25:25.005 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215107 -lt 103079215107 2026-02-17T12:25:25.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:25:25.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:25:25.006 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:25:25.258 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:25:25.444 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:25:25.444 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:25:25.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:25:25.445 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:338: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-17T12:25:25.697 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-92877-0 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-92877-0 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-92877-0 2026-02-17T12:25:25.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:25:25.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-92877-0 td/test-erasure-eio/ORIGINAL 2026-02-17T12:25:25.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-0 td/test-erasure-eio 0 2026-02-17T12:25:25.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:25:25.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:25:25.717 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-0 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-0 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:25:25.718 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-0 2026-02-17T12:25:25.719 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-0 2026-02-17T12:25:25.719 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:25:25.910 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:25:25.910 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:25:25.910 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=3 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:25:25.911 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/3/type 2026-02-17T12:25:25.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:25:25.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-17T12:25:25.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:25:25.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=3 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.3 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:25:25.914 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:25:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:25:25.960 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.3 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.3 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.3 ']' 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:25.961 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:25.962 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:25.962 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.3.asok 2026-02-17T12:25:25.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:25:25.962 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-92877-0 0 2026-02-17T12:25:26.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-0 2026-02-17T12:25:26.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:25:26.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:25:26.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-0 2026-02-17T12:25:26.008 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:25:26.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:25:26.009 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-92877-0 td/test-erasure-eio/COPY 2026-02-17T12:25:26.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:25:26.028 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:25:26.029 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-17T12:25:26.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-92877-0 2026-02-17T12:25:26.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:25:26.030 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-0 2026-02-17T12:25:26.031 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-0 2026-02-17T12:25:26.031 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:25:26.231 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:25:26.232 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:25:26.233 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:25:27.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:25:27.540 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-17T12:25:27.783 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already out. 2026-02-17T12:25:27.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-92877-0 2026-02-17T12:25:27.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-17T12:25:27.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:25:27.792 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-0 2026-02-17T12:25:27.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-0 2026-02-17T12:25:27.793 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:25:27.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:25:27.984 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:25:27.984 INFO:tasks.workunit.client.0.trial044.stderr:0' 2026-02-17T12:25:27.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 0 2026-02-17T12:25:27.984 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-17T12:25:28.189 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already in. 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:28.197 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:25:28.198 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:25:28.199 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:25:28.200 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:25:28.200 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:28.200 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:25:28.201 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:25:28.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:25:28.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:25:28.202 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:25:28.216 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:28.215+0000 7f2ee0a99780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:28.217 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:28.216+0000 7f2ee0a99780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:28.218 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:28.217+0000 7f2ee0a99780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:28.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:25:28.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:28.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:25:28.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:28.395 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:28.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:28.396 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:28.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:28.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:28.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:28.584 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:29.558 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:29.556+0000 7f2ee0a99780 -1 Falling back to public interface 2026-02-17T12:25:29.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:29.585 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:29.585 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:29.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:29.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:29.586 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:29.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:30.781 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:30.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:30.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:25:30.782 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:30.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:30.782 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:30.970 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:31.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:31.972 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:31.972 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:31.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:31.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:31.973 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:32.160 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:32.519 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:32.517+0000 7f2ee0a99780 -1 osd.2 31 log_to_monitors true 2026-02-17T12:25:33.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:33.161 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:33.161 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:33.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:33.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:33.162 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 40 up_thru 17 down_at 32 last_clean_interval [17,31) [v2:127.0.0.1:6818/3933344164,v1:127.0.0.1:6819/3933344164] [v2:127.0.0.1:6820/3933344164,v1:127.0.0.1:6821/3933344164] exists,up faf4900f-5452-4707-8490-ef9b0edb5437 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:25:33.350 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:25:33.351 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:25:33.351 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:25:33.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:25:33.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:25:33.351 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:25:33.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:25:33.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:25:33.439 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:25:33.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:25:33.440 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:25:33.440 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:25:33.626 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:33.627 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:25:33.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803789 2026-02-17T12:25:33.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803789 2026-02-17T12:25:33.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789' 2026-02-17T12:25:33.691 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:33.691 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:25:33.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607563 2026-02-17T12:25:33.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607563 2026-02-17T12:25:33.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563' 2026-02-17T12:25:33.757 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:33.757 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:25:33.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691843 2026-02-17T12:25:33.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691843 2026-02-17T12:25:33.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-171798691843' 2026-02-17T12:25:33.819 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:25:33.819 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:25:33.882 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215111 2026-02-17T12:25:33.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215111 2026-02-17T12:25:33.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803789 1-51539607563 2-171798691843 3-103079215111' 2026-02-17T12:25:33.883 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:33.883 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803789 2026-02-17T12:25:33.883 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:33.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:25:33.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803789 2026-02-17T12:25:33.885 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:33.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803789 2026-02-17T12:25:33.886 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-17T12:25:33.886 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803789 2026-02-17T12:25:33.887 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:34.078 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:25:34.079 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:25:35.080 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:25:35.080 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:35.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-17T12:25:35.268 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:25:36.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:25:36.270 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:25:36.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-17T12:25:36.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:36.460 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607563 2026-02-17T12:25:36.460 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:36.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:25:36.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607563 2026-02-17T12:25:36.461 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:36.462 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607563 2026-02-17T12:25:36.462 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607563 2026-02-17T12:25:36.463 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607563' 2026-02-17T12:25:36.463 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:25:36.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607563 -lt 51539607563 2026-02-17T12:25:36.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:36.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691843 2026-02-17T12:25:36.657 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:36.658 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:25:36.658 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691843 2026-02-17T12:25:36.659 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:36.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691843 2026-02-17T12:25:36.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691843' 2026-02-17T12:25:36.660 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 171798691843 2026-02-17T12:25:36.660 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:25:36.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-17T12:25:36.854 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:25:36.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-103079215111 2026-02-17T12:25:36.854 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:25:36.855 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:25:36.856 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-103079215111 2026-02-17T12:25:36.856 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:25:36.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215111 2026-02-17T12:25:36.857 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 103079215111' 2026-02-17T12:25:36.857 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 103079215111 2026-02-17T12:25:36.858 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:25:37.046 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215112 -lt 103079215111 2026-02-17T12:25:37.047 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:25:37.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:25:37.048 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:25:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:25:37.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:25:37.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:25:37.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:25:37.487 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:25:37.487 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:25:37.487 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:25:37.746 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:25:37.956 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:25:37.963 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:25:38.164 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:25:38.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:25:38.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:25:38.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:25:38.171 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:25:38.171 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:25:38.172 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:25:38.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:25:38.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:25:38.172 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:25:38.280 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:25:38.281 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:25:38.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:25:38.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:25:38.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:25:38.284 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:25:38.285 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:25:38.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:25:38.285 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:25:38.286 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:25:38.286 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:25:38.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:25:38.287 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:25:38.288 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:25:38.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:25:38.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.299 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.299 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:25:38.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:25:38.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/test-erasure-eio 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/test-erasure-eio 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:25:38.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:25:38.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:25:38.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:25:38.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:25:38.303 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:25:38.304 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:25:38.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:25:38.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:25:38.305 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:25:38.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:25:38.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:25:38.306 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:25:38.306 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:25:38.307 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:25:38.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:25:38.308 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:25:38.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:25:38.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.309 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.309 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:25:38.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:25:38.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:25:38.310 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/test-erasure-eio 2026-02-17T12:25:38.311 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:25:38.312 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.312 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.312 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92877 2026-02-17T12:25:38.313 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/test-erasure-eio 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:25:38.314 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:25:38.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:25:38.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/test-erasure-eio/a 2026-02-17T12:25:38.315 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.334 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:38.335 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:25:38.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:25:38.360 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:25:38.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:25:38.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:25:38.361 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:38.362 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.363 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.363 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:25:38.363 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:25:38.363 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get fsid 2026-02-17T12:25:38.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:25:38.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:25:38.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:25:38.405 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.406 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:25:38.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:25:38.407 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92877/ceph-mon.a.asok config get mon_host 2026-02-17T12:25:38.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-17T12:25:38.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/test-erasure-eio 2026-02-17T12:25:38.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:25:38.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:25:38.450 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:25:38.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/test-erasure-eio/x 2026-02-17T12:25:38.451 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:25:38.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:25:38.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:38.548 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:38.548 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:38.549 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:38.549 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:38.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:38.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:25:38.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:25:38.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-17T12:25:38.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:25:38.665 INFO:tasks.workunit.client.0.trial044.stderr:pool 'rbd' already exists 2026-02-17T12:25:38.672 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:25:39.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-17T12:25:39.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:25:39.675 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:25:39.675 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:25:39.676 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:39.676 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:39.676 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-mon.a.asok 2026-02-17T12:25:39.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-17T12:25:39.676 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-mon.a.asok log flush 2026-02-17T12:25:39.718 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:25:39.722 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-17T12:25:39.724 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:25:38.351+0000 7f47ce3d0d80 0 load: jerasure load: lrc 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:347: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-17T12:25:39.725 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-17T12:25:39.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-17T12:25:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:25:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-17T12:25:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:39.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:39.730 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:25:39.731 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d833e7f6-8111-43eb-b4ae-4d86c3b9461b 2026-02-17T12:25:39.732 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 d833e7f6-8111-43eb-b4ae-4d86c3b9461b' 2026-02-17T12:25:39.733 INFO:tasks.workunit.client.0.trial044.stdout:add osd0 d833e7f6-8111-43eb-b4ae-4d86c3b9461b 2026-02-17T12:25:39.733 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:25:39.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBDXpRpbf0gLBAANb5E6J2czMfK2TZwMWW1fA== 2026-02-17T12:25:39.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBDXpRpbf0gLBAANb5E6J2czMfK2TZwMWW1fA=="}' 2026-02-17T12:25:39.742 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d833e7f6-8111-43eb-b4ae-4d86c3b9461b -i td/test-erasure-eio/0/new.json 2026-02-17T12:25:39.843 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:39.849 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-17T12:25:39.850 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDXpRpbf0gLBAANb5E6J2czMfK2TZwMWW1fA== --osd-uuid d833e7f6-8111-43eb-b4ae-4d86c3b9461b 2026-02-17T12:25:39.867 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:39.865+0000 7f8e12410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:39.869 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:39.867+0000 7f8e12410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:39.870 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:39.868+0000 7f8e12410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:39.870 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:39.869+0000 7f8e12410780 -1 bdev(0x5558a1721c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:25:39.870 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:39.869+0000 7f8e12410780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-17T12:25:42.252 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-17T12:25:42.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:25:42.253 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:25:42.254 INFO:tasks.workunit.client.0.trial044.stdout:adding osd0 key to auth repository 2026-02-17T12:25:42.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:25:42.526 INFO:tasks.workunit.client.0.trial044.stdout:start osd.0 2026-02-17T12:25:42.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:25:42.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:42.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:25:42.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:25:42.528 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:25:42.543 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:42.542+0000 7fa6f89b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:42.545 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:42.543+0000 7fa6f89b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:42.545 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:42.544+0000 7fa6f89b4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:42.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:25:42.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:42.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:25:42.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:42.725 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:42.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:42.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:42.726 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:42.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:42.727 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:25:42.911 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:43.623 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:43.622+0000 7fa6f89b4780 -1 Falling back to public interface 2026-02-17T12:25:43.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:43.912 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:43.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:43.913 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:43.913 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:43.914 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:25:44.101 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:45.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:45.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:45.103 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:25:45.103 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:45.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:45.104 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:25:45.290 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:46.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:46.291 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:46.292 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:46.292 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:46.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:46.293 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:25:46.343 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:46.342+0000 7fa6f89b4780 -1 osd.0 0 log_to_monitors true 2026-02-17T12:25:46.482 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:47.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:47.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:47.485 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:47.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:47.485 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:47.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:25:47.677 INFO:tasks.workunit.client.0.trial044.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4070985208,v1:127.0.0.1:6803/4070985208] [v2:127.0.0.1:6804/4070985208,v1:127.0.0.1:6805/4070985208] exists,up d833e7f6-8111-43eb-b4ae-4d86c3b9461b 2026-02-17T12:25:47.681 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:25:47.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:25:47.683 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:47.684 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:25:47.685 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c2ac27f3-495e-45bb-95ce-2d9a96c39a51 2026-02-17T12:25:47.686 INFO:tasks.workunit.client.0.trial044.stdout:add osd1 c2ac27f3-495e-45bb-95ce-2d9a96c39a51 2026-02-17T12:25:47.686 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 c2ac27f3-495e-45bb-95ce-2d9a96c39a51' 2026-02-17T12:25:47.687 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:25:47.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBLXpRpM+M6KRAAgQJAla/g2J1j7CZ7EQ74og== 2026-02-17T12:25:47.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBLXpRpM+M6KRAAgQJAla/g2J1j7CZ7EQ74og=="}' 2026-02-17T12:25:47.693 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c2ac27f3-495e-45bb-95ce-2d9a96c39a51 -i td/test-erasure-eio/1/new.json 2026-02-17T12:25:47.893 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:47.899 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-17T12:25:47.900 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLXpRpM+M6KRAAgQJAla/g2J1j7CZ7EQ74og== --osd-uuid c2ac27f3-495e-45bb-95ce-2d9a96c39a51 2026-02-17T12:25:47.918 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:47.917+0000 7fcba5d2a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:47.919 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:47.918+0000 7fcba5d2a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:47.920 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:47.919+0000 7fcba5d2a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:47.921 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:47.919+0000 7fcba5d2a780 -1 bdev(0x55978f8b1c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:25:47.921 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:47.919+0000 7fcba5d2a780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-17T12:25:50.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-17T12:25:50.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:25:50.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:25:50.066 INFO:tasks.workunit.client.0.trial044.stdout:adding osd1 key to auth repository 2026-02-17T12:25:50.066 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:25:50.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:25:50.320 INFO:tasks.workunit.client.0.trial044.stdout:start osd.1 2026-02-17T12:25:50.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:50.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:25:50.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:25:50.322 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:25:50.336 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:50.335+0000 7f51983f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:50.337 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:50.336+0000 7f51983f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:50.338 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:50.337+0000 7f51983f9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:50.514 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:50.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:50.515 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:50.515 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:50.516 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:50.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:51.674 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:51.673+0000 7f51983f9780 -1 Falling back to public interface 2026-02-17T12:25:51.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:51.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:51.702 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:51.702 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:51.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:51.703 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:51.895 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:52.896 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:53.085 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:54.086 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:54.276 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:54.374 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:54.372+0000 7f51983f9780 -1 osd.1 0 log_to_monitors true 2026-02-17T12:25:55.277 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:55.278 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:55.278 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:25:55.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:25:55.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:55.279 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:25:55.470 INFO:tasks.workunit.client.0.trial044.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1137259716,v1:127.0.0.1:6811/1137259716] [v2:127.0.0.1:6812/1137259716,v1:127.0.0.1:6813/1137259716] exists,up c2ac27f3-495e-45bb-95ce-2d9a96c39a51 2026-02-17T12:25:55.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:25:55.475 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:25:55.476 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:25:55.477 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=dc86c620-e700-4492-aeed-a514e2c4c73d 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 dc86c620-e700-4492-aeed-a514e2c4c73d' 2026-02-17T12:25:55.478 INFO:tasks.workunit.client.0.trial044.stdout:add osd2 dc86c620-e700-4492-aeed-a514e2c4c73d 2026-02-17T12:25:55.479 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:25:55.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTXpRpAYjlHBAAL0UtpB2k3d8XmpVA1Xtb/Q== 2026-02-17T12:25:55.486 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTXpRpAYjlHBAAL0UtpB2k3d8XmpVA1Xtb/Q=="}' 2026-02-17T12:25:55.487 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new dc86c620-e700-4492-aeed-a514e2c4c73d -i td/test-erasure-eio/2/new.json 2026-02-17T12:25:55.680 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:25:55.687 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-17T12:25:55.688 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTXpRpAYjlHBAAL0UtpB2k3d8XmpVA1Xtb/Q== --osd-uuid dc86c620-e700-4492-aeed-a514e2c4c73d 2026-02-17T12:25:55.705 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:55.704+0000 7f3ae5a0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:55.706 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:55.705+0000 7f3ae5a0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:55.707 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:55.706+0000 7f3ae5a0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:55.708 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:55.707+0000 7f3ae5a0d780 -1 bdev(0x556a81d23c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:25:55.708 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:55.707+0000 7f3ae5a0d780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-17T12:25:58.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-17T12:25:58.068 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:25:58.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:25:58.069 INFO:tasks.workunit.client.0.trial044.stdout:adding osd2 key to auth repository 2026-02-17T12:25:58.069 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:25:58.320 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:25:58.321 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:25:58.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:25:58.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:25:58.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:25:58.321 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:25:58.337 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:58.336+0000 7f9675011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:58.339 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:58.338+0000 7f9675011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:58.340 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:58.339+0000 7f9675011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:58.509 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:58.698 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:59.699 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:25:59.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:25:59.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:25:59.700 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:25:59.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:25:59.700 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:25:59.885 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:25:59.926 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:25:59.924+0000 7f9675011780 -1 Falling back to public interface 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:00.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:01.076 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:02.077 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:02.269 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:02.876 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:02.873+0000 7f9675011780 -1 osd.2 0 log_to_monitors true 2026-02-17T12:26:03.270 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:03.271 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:03.271 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:26:03.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:26:03.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:03.272 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:03.459 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:04.460 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:04.461 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:26:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:26:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:04.461 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:04.651 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/322700523,v1:127.0.0.1:6819/322700523] [v2:127.0.0.1:6820/322700523,v1:127.0.0.1:6821/322700523] exists,up dc86c620-e700-4492-aeed-a514e2c4c73d 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/test-erasure-eio 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:26:04.653 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:26:04.654 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:26:04.655 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=78300a6d-b68a-4676-8e2c-23be812cf5b6 2026-02-17T12:26:04.656 INFO:tasks.workunit.client.0.trial044.stdout:add osd3 78300a6d-b68a-4676-8e2c-23be812cf5b6 2026-02-17T12:26:04.656 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 78300a6d-b68a-4676-8e2c-23be812cf5b6' 2026-02-17T12:26:04.656 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:26:04.666 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBcXpRp1dSlJxAAjZER7dyHbfGXvZQtn1EwMw== 2026-02-17T12:26:04.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBcXpRp1dSlJxAAjZER7dyHbfGXvZQtn1EwMw=="}' 2026-02-17T12:26:04.667 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 78300a6d-b68a-4676-8e2c-23be812cf5b6 -i td/test-erasure-eio/3/new.json 2026-02-17T12:26:04.856 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:26:04.863 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-17T12:26:04.864 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcXpRp1dSlJxAAjZER7dyHbfGXvZQtn1EwMw== --osd-uuid 78300a6d-b68a-4676-8e2c-23be812cf5b6 2026-02-17T12:26:04.883 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:04.880+0000 7fefbb13f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:04.884 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:04.882+0000 7fefbb13f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:04.885 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:04.882+0000 7fefbb13f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:04.885 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:04.883+0000 7fefbb13f780 -1 bdev(0x55ff5c2b7c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:26:04.886 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:04.883+0000 7fefbb13f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-17T12:26:06.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-17T12:26:06.993 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:26:06.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:26:06.994 INFO:tasks.workunit.client.0.trial044.stdout:adding osd3 key to auth repository 2026-02-17T12:26:06.994 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:26:07.253 INFO:tasks.workunit.client.0.trial044.stdout:start osd.3 2026-02-17T12:26:07.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:26:07.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:26:07.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:26:07.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:26:07.254 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:26:07.270 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:07.267+0000 7feceee1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:07.272 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:07.270+0000 7feceee1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:07.273 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:07.270+0000 7feceee1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:26:07.442 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:26:07.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:07.443 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:07.632 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:08.633 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:08.823 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:08.865 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:08.863+0000 7feceee1d780 -1 Falling back to public interface 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:09.824 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:10.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:11.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:11.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:11.014 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:26:11.015 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:26:11.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:11.015 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:11.203 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:11.819 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:11.818+0000 7feceee1d780 -1 osd.3 0 log_to_monitors true 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:12.204 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:12.396 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:13.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:13.397 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:13.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:26:13.398 INFO:tasks.workunit.client.0.trial044.stdout:5 2026-02-17T12:26:13.398 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:13.399 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:26:13.585 INFO:tasks.workunit.client.0.trial044.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3820281793,v1:127.0.0.1:6827/3820281793] [v2:127.0.0.1:6828/3820281793,v1:127.0.0.1:6829/3820281793] exists,up 78300a6d-b68a-4676-8e2c-23be812cf5b6 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:26:13.587 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.0.asok 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-17T12:26:13.588 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.0.asok log flush 2026-02-17T12:26:13.631 INFO:tasks.workunit.client.0.trial044.stdout:{} 2026-02-17T12:26:13.635 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stdout:2026-02-17T12:25:43.626+0000 7fa6f89b4780 0 load: jerasure load: lrc 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:349: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-17T12:26:13.637 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-17T12:26:13.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-17T12:26:13.887 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-17T12:26:14.182 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' already exists 2026-02-17T12:26:14.190 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:26:15.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-17T12:26:15.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:26:15.191 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:26:15.192 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:26:15.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:26:15.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:26:15.192 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:26:15.192 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:26:15.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:26:15.193 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:26:15.282 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:15.474 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:26:15.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803784 2026-02-17T12:26:15.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803784 2026-02-17T12:26:15.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784' 2026-02-17T12:26:15.539 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:15.539 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:26:15.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607559 2026-02-17T12:26:15.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607559 2026-02-17T12:26:15.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-17T12:26:15.602 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:15.603 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:26:15.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411333 2026-02-17T12:26:15.662 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411333 2026-02-17T12:26:15.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411333' 2026-02-17T12:26:15.663 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:15.663 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:26:15.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T12:26:15.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T12:26:15.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411333 3-111669149699' 2026-02-17T12:26:15.726 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:15.726 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803784 2026-02-17T12:26:15.727 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:15.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:26:15.728 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803784 2026-02-17T12:26:15.728 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:15.728 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803784 2026-02-17T12:26:15.729 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-17T12:26:15.729 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803784 2026-02-17T12:26:15.729 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:26:15.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-17T12:26:15.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:26:16.923 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:26:16.924 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:26:17.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-17T12:26:17.112 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:17.113 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607559 2026-02-17T12:26:17.113 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:17.113 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:26:17.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607559 2026-02-17T12:26:17.114 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:17.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607559 2026-02-17T12:26:17.115 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-17T12:26:17.116 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607559 2026-02-17T12:26:17.116 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:26:17.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-17T12:26:17.301 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:17.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-77309411333 2026-02-17T12:26:17.302 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:17.302 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:26:17.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-77309411333 2026-02-17T12:26:17.303 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:17.304 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411333 2026-02-17T12:26:17.304 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 77309411333 2026-02-17T12:26:17.305 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-17T12:26:17.305 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:26:17.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-17T12:26:17.491 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:17.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149699 2026-02-17T12:26:17.491 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:17.492 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:26:17.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149699 2026-02-17T12:26:17.493 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:17.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T12:26:17.494 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-17T12:26:17.494 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149699 2026-02-17T12:26:17.494 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:26:17.682 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T12:26:17.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:26:17.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:26:17.683 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:26:17.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:26:17.936 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:26:17.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:26:17.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:26:17.936 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:26:17.937 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:26:17.937 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:26:17.937 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:26:18.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:26:18.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:26:18.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:26:18.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:352: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-92877-1 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-92877-1 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-17T12:26:18.382 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-17T12:26:18.383 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-92877-1 td/test-erasure-eio/ORIGINAL 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-92877-1 td/test-erasure-eio 1 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=ec 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=pool-jerasure 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=obj-eio-92877-1 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/test-erasure-eio 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:26:18.402 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-1 2026-02-17T12:26:18.403 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.403 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:26:18.591 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('3' '1' '2') 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' ec '!=' ec ']' 2026-02-17T12:26:18.592 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/test-erasure-eio/1/type 2026-02-17T12:26:18.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:26:18.594 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:18.595 INFO:tasks.workunit.client.0.trial044.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:18.596 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:26:18.596 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.92877/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:26:18.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:26:18.644 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:26:18.645 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:26:18.645 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92877/ceph-osd.1.asok 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:26:18.646 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.92877/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-92877-1 1 2026-02-17T12:26:18.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.694 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-17T12:26:18.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-17T12:26:18.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-92877-1 2026-02-17T12:26:18.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-17T12:26:18.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-17T12:26:18.695 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-92877-1 td/test-erasure-eio/COPY 2026-02-17T12:26:18.712 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-17T12:26:18.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-17T12:26:18.713 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-17T12:26:18.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:26:18.714 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-1 2026-02-17T12:26:18.715 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-1 2026-02-17T12:26:18.715 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:26:18.906 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:2' 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:26:18.907 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:26:18.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:26:18.908 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:26:20.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:26:20.216 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-17T12:26:20.458 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already out. 2026-02-17T12:26:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-92877-1 2026-02-17T12:26:20.467 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=pool-jerasure 2026-02-17T12:26:20.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-17T12:26:20.468 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=obj-eio-92877-1 2026-02-17T12:26:20.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map pool-jerasure obj-eio-92877-1 2026-02-17T12:26:20.469 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:26:20.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T12:26:20.660 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:26:20.660 INFO:tasks.workunit.client.0.trial044.stderr:0' 2026-02-17T12:26:20.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 0 2026-02-17T12:26:20.660 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-17T12:26:20.862 INFO:tasks.workunit.client.0.trial044.stderr:osd.2 is already in. 2026-02-17T12:26:20.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-17T12:26:20.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/test-erasure-eio 2026-02-17T12:26:20.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:26:20.872 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-17T12:26:20.873 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:26:20.874 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:26:20.875 INFO:tasks.workunit.client.0.trial044.stdout:start osd.2 2026-02-17T12:26:20.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=11ce7075-56ad-4bbe-a05a-c67b1c2c97d3 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.92877/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:26:20.876 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-17T12:26:20.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:26:20.876 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:26:20.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:26:20.877 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:26:20.888 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:20.886+0000 7f94214d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:20.889 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:20.887+0000 7f94214d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:20.890 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:20.888+0000 7f94214d5780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stdout:0 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:21.063 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:21.255 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:22.231 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:22.230+0000 7f94214d5780 -1 Falling back to public interface 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stdout:1 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:22.256 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:22.447 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:23.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:23.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:23.448 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:26:23.449 INFO:tasks.workunit.client.0.trial044.stdout:2 2026-02-17T12:26:23.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:23.449 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:23.641 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stdout:3 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:24.642 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:24.830 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:26:24.917 INFO:tasks.workunit.client.0.trial044.stderr:2026-02-17T12:26:24.916+0000 7f94214d5780 -1 osd.2 33 log_to_monitors true 2026-02-17T12:26:25.831 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:26:25.832 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:26:25.832 INFO:tasks.workunit.client.0.trial044.stdout:4 2026-02-17T12:26:25.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:26:25.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:26:25.833 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:26:26.022 INFO:tasks.workunit.client.0.trial044.stdout:osd.2 up in weight 1 up_from 42 up_thru 20 down_at 34 last_clean_interval [18,33) [v2:127.0.0.1:6818/544126037,v1:127.0.0.1:6819/544126037] [v2:127.0.0.1:6820/544126037,v1:127.0.0.1:6821/544126037] exists,up dc86c620-e700-4492-aeed-a514e2c4c73d 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:26:26.024 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:26:26.025 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:26:26.110 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:1 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:2 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:3' 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:26.296 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:26:26.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=25769803788 2026-02-17T12:26:26.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 25769803788 2026-02-17T12:26:26.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788' 2026-02-17T12:26:26.359 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:26.359 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:26:26.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607563 2026-02-17T12:26:26.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607563 2026-02-17T12:26:26.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563' 2026-02-17T12:26:26.420 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:26.420 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:26:26.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626435 2026-02-17T12:26:26.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626435 2026-02-17T12:26:26.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-180388626435' 2026-02-17T12:26:26.484 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:26:26.485 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149703 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149703 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-180388626435 3-111669149703' 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-25769803788 2026-02-17T12:26:26.545 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:26.547 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:26:26.547 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-25769803788 2026-02-17T12:26:26.547 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:26.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=25769803788 2026-02-17T12:26:26.548 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-17T12:26:26.548 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.0 seq 25769803788 2026-02-17T12:26:26.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:26:26.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-17T12:26:26.737 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:26.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-51539607563 2026-02-17T12:26:26.738 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:26.738 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:26:26.739 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-51539607563 2026-02-17T12:26:26.739 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:26.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607563 2026-02-17T12:26:26.741 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 51539607563' 2026-02-17T12:26:26.741 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.1 seq 51539607563 2026-02-17T12:26:26.741 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:26:26.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607563 -lt 51539607563 2026-02-17T12:26:26.931 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:26.932 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-180388626435 2026-02-17T12:26:26.932 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:26.933 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:26:26.933 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-180388626435 2026-02-17T12:26:26.934 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:26.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626435 2026-02-17T12:26:26.935 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-17T12:26:26.935 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.2 seq 180388626435 2026-02-17T12:26:26.935 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:26:27.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-17T12:26:27.124 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:26:27.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-111669149703 2026-02-17T12:26:27.125 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:26:27.125 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:26:27.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-111669149703 2026-02-17T12:26:27.126 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:26:27.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149703 2026-02-17T12:26:27.127 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-17T12:26:27.128 INFO:tasks.workunit.client.0.trial044.stdout:waiting osd.3 seq 111669149703 2026-02-17T12:26:27.128 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:26:27.316 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-17T12:26:27.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:26:27.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:26:27.317 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:26:27.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:26:27.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:26:27.756 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:26:27.756 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:26:27.757 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:26:27.757 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:26:28.012 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:26:28.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:26:28.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:26:28.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:354: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-17T12:26:28.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-17T12:26:28.013 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-17T12:26:28.224 INFO:tasks.workunit.client.0.trial044.stderr:pool 'pool-jerasure' does not exist 2026-02-17T12:26:28.231 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-17T12:26:28.430 INFO:tasks.workunit.client.0.trial044.stderr:erasure-code-profile myprofile does not exist 2026-02-17T12:26:28.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-17T12:26:28.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:26:28.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:26:28.437 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:26:28.438 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:26:28.438 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:26:28.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:26:28.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:26:28.438 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:26:28.549 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:26:28.549 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:26:28.550 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:26:28.550 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:26:28.551 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:26:28.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:26:28.552 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:26:28.552 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:26:28.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:26:28.553 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:26:28.553 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:26:28.554 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:26:28.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:26:28.555 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:26:28.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:26:28.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:28.565 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:28.565 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:26:28.566 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/test-erasure-eio 0 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/test-erasure-eio 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:26:28.567 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:26:28.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:26:28.568 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:26:28.569 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:26:28.569 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:26:28.570 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:26:28.570 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:26:28.571 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:26:28.571 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:26:28.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:26:28.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:26:28.572 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:26:28.573 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:26:28.573 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:26:28.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:26:28.574 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/test-erasure-eio 2026-02-17T12:26:28.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:26:28.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:26:28.575 INFO:tasks.workunit.client.0.trial044.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92877 2026-02-17T12:26:28.575 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92877 2026-02-17T12:26:28.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:26:28.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:26:28.577 INFO:tasks.workunit.client.0.trial044.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:26:28.577 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:26:28.578 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:26:28.605 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-17T12:26:28.605 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T12:26:28.951 DEBUG:teuthology.parallel:result is None 2026-02-17T12:26:28.951 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:26:28.976 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:26:28.976 DEBUG:teuthology.orchestra.run.trial044:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T12:26:29.032 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:26:29.032 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T12:26:29.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T12:26:29.045 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T12:26:29.200 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T12:26:29.214 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T12:26:29.214 DEBUG:teuthology.orchestra.run.trial044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T12:26:29.230 INFO:teuthology.orchestra.run.trial044.stderr:bash: line 1: ntpq: command not found 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:=============================================================================== 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:^+ 216.82.35.115 2 7 377 56 +2454us[+2454us] +/- 31ms 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:^+ ntp2.wiktel.com 1 6 377 52 -1304us[-1304us] +/- 20ms 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:^- kc1cloud.147.72.155.23.m> 3 7 377 50 -277us[ -277us] +/- 54ms 2026-02-17T12:26:29.356 INFO:teuthology.orchestra.run.trial044.stdout:^* 1636541-static.urdliaaa.> 1 7 377 117 -44us[ -54us] +/- 15ms 2026-02-17T12:26:29.357 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T12:26:29.368 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T12:26:29.368 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T12:26:29.383 DEBUG:teuthology.orchestra.run.trial044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T12:26:29.451 DEBUG:teuthology.orchestra.run.trial044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T12:26:29.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T12:26:29.476 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T12:26:29.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T12:26:29.505 INFO:teuthology.task.internal:Duration was 1718.609175 seconds 2026-02-17T12:26:29.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T12:26:29.523 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T12:26:29.523 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T12:26:29.559 INFO:teuthology.orchestra.run.trial044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T12:26:29.737 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T12:26:29.737 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial044.front.sepia.ceph.com 2026-02-17T12:26:29.737 DEBUG:teuthology.orchestra.run.trial044:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T12:26:29.797 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T12:26:29.797 DEBUG:teuthology.orchestra.run.trial044:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T12:26:29.897 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T12:26:29.897 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T12:26:29.921 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T12:26:29.921 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T12:26:29.922 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T12:26:29.922 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T12:26:29.922 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T12:26:29.931 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T12:26:29.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T12:26:29.950 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T12:26:29.950 DEBUG:teuthology.orchestra.run.trial044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T12:26:29.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T12:26:30.016 DEBUG:teuthology.orchestra.run.trial044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:26:30.065 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern = core 2026-02-17T12:26:30.077 DEBUG:teuthology.orchestra.run.trial044:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:26:30.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T12:26:30.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T12:26:30.149 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T12:26:30.151 DEBUG:teuthology.misc:Transferring archived files from trial044:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51384/remote/trial044 2026-02-17T12:26:30.153 DEBUG:teuthology.orchestra.run.trial044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T12:26:30.212 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T12:26:30.213 DEBUG:teuthology.orchestra.run.trial044:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T12:26:30.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T12:26:30.278 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T12:26:30.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T12:26:30.295 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T12:26:30.296 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T12:26:30.317 INFO:teuthology.orchestra.run.trial044.stdout: 791885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 12:26 /home/ubuntu/cephtest 2026-02-17T12:26:30.318 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T12:26:30.335 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T12:26:30.372 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} duration: 1718.6091754436493 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-17T12:26:30.372 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T12:26:30.498 INFO:teuthology.run:pass